twmarketdata 0.1.0__py3-none-any.whl

twmarketdata-0.1.0.dist-info/METADATA

@@ -0,0 +1,319 @@
+Metadata-Version: 2.4
+Name: twmarketdata
+Version: 0.1.0
+Summary: Data-access client for the TW Market Data API. Retrieval only, for research and education.
+Project-URL: Homepage, https://twmarketdata.com
+Project-URL: Documentation, https://api.twmarketdata.com/openapi.json
+Author: TW Market Data
+License: MIT License
+        
+        Copyright (c) 2026 TW Market Data
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Keywords: api-client,dataset,market-data,research,taiwan
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Education
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Database :: Front-Ends
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Typing :: Typed
+Requires-Python: >=3.9
+Requires-Dist: httpx<1.0,>=0.27
+Requires-Dist: pandas>=1.5
+Provides-Extra: test
+Requires-Dist: pytest>=7.4; extra == 'test'
+Requires-Dist: respx>=0.21; extra == 'test'
+Description-Content-Type: text/markdown
+
+# twmd
+
+Python data-access client for the [TW Market Data](https://twmarketdata.com) API.
+
+`twmd` retrieves published datasets over HTTP and returns them as pandas
+DataFrames. It is a transport layer for data retrieval, intended for research
+and educational use. It fetches records as the API publishes them and performs
+no analysis, scoring, ranking, or interpretation of any kind. Deciding what the
+data means, and what if anything to do about it, is entirely the caller's own
+work and responsibility.
+
+Responses carry the API's own `lineage.not_investment_advice` flag; this client
+preserves it unmodified.
+
+## Install
+
+```bash
+pip install twmarketdata
+```
+
+The distribution is named `twmarketdata`; the import package is `twmd`:
+
+```python
+import twmd
+```
+
+Pure Python. Dependencies are `httpx` and `pandas` — no compiled extensions, no
+system libraries, installable in a restricted sandbox.
+
+Requires Python 3.9 or newer. The floors are deliberately low (`pandas>=1.5`,
+`httpx>=0.27`) so this installs into existing environments without forcing an
+upgrade, and CI runs the full suite pinned to exactly those floors so they stay
+honest rather than aspirational.
+
+One caveat that is not ours to fix: pandas 1.5 wheels are built against the
+numpy 1.x C ABI and fail at import under numpy 2 with `numpy.dtype size
+changed`. If you are pinned to pandas 1.x, pin `numpy<2` alongside it. Anything
+from pandas 2.2.2 onward works with numpy 2 unconstrained.
+
+## Quick start — no API key needed
+
+Five sample tickers are served without credentials on selected datasets, so
+this runs with zero configuration:
+
+```python
+from twmd import Client
+
+client = Client()
+df = client.get_dataset("twse-daily-price", symbol="2330", limit=5)
+
+print(df[["date", "open", "high", "low", "close", "volume_shares"]])
+print(df.attrs["data_as_of"])       # data freshness date
+print(df.attrs["lineage"])          # provider, source endpoints, source table
+```
+
+## Authentication
+
+Set your key in the environment; it is never written to disk or logged:
+
+```bash
+export TWMD_API_KEY="sk_live_..."
+```
+
+`Client()` picks it up automatically. You may also pass `Client(api_key=...)`
+explicitly. With no key set, the client runs in key-free mode and reaches the
+datasets listed below.
+
+## Key-free access matrix
+
+Key-free access is scoped **per dataset**, not globally per ticker. The same
+ticker that is served on one dataset may require a key on another. Measured
+against the live API on 2026-07-21:
+
+| Tier | Datasets | Tickers served without a key |
+| --- | --- | --- |
+| Open | `security-master`, `market-index` | any |
+| Sample | `twse-daily-price`, `tpex-daily-price`, `monthly-revenue` | `2330`, `2317`, `2454`, `0050`, `2603` only |
+| Key required | everything else, including `institutional-flow`, `market-prices`, `financial-metrics`, `income-statement`, `balance-sheet` | none |
+
+Check before requesting:
+
+```python
+from twmd import is_key_free
+
+is_key_free("twse-daily-price", "2330")     # True
+is_key_free("twse-daily-price", "1101")     # False — outside the sample set
+is_key_free("security-master", "1101")      # True — open dataset
+is_key_free("institutional-flow", "2330")   # False — key required
+```
+
+Datasets not in the table are treated as key-required, which is the safe
+default. The client never blocks a request on this basis — it only uses the
+matrix to explain a 401 after the fact.
+
+## DataFrames and metadata
+
+Records become rows. Everything else in the response envelope is preserved on
+`df.attrs`.
+
+The API uses **two envelope shapes**, and `twmd` handles both transparently:
+
+```python
+# rows / count — twse-daily-price, tpex-daily-price, monthly-revenue
+df.attrs["dataset"]       # "twse_daily_price"
+df.attrs["count"]         # record count
+df.attrs["data_as_of"]    # data freshness date
+df.attrs["source_role"]   # e.g. "official_twse"
+df.attrs["lineage"]       # provider, official_source, source_endpoints, table
+df.attrs["meta"]          # last_trading_day, market_status
+
+# items / row_count — security-master, market-index
+df.attrs["dataset_id"]                  # "security-master"
+df.attrs["row_count"]                   # record count
+df.attrs["as_of_date"]                  # snapshot date
+df.attrs["survivorship_bias_warning"]   # integrity caveat raised by the API
+```
+
+Envelope contents vary by dataset — `monthly-revenue` sends only `dataset`,
+`rows` and `count` — so read `attrs` defensively.
+
+Records in the `items` variant contain nested objects (`security_identity`,
+`market_identity`, `index_level`). These stay as dict-valued columns rather than
+being flattened, so a DataFrame reflects what the API actually sent. Expand one
+when you want to:
+
+```python
+import pandas as pd
+identity = pd.json_normalize(df["security_identity"])
+```
+
+Note that `security-master` carries a `survivorship_bias_warning` stating the
+current master is not point-in-time complete. It is surfaced on `attrs`
+unmodified; check it before using that dataset for historical work.
+
+## Errors
+
+```python
+from twmd import Client, TwmdAuthError, TwmdPaymentRequired
+
+try:
+    df = client.get_dataset("institutional-flow", symbol="2330")
+except TwmdAuthError as exc:
+    print(exc.error_code)   # "missing_api_key" or "invalid_api_key"
+    print(exc.body)         # decoded response body, verbatim
+```
+
+Every `TwmdAPIError` subclass — every row in the table below except the last —
+exposes `.status_code`, `.body` (decoded body, unmodified), `.text` and
+`.error_code`. `TwmdTransportError` and `TwmdConfigError` derive from `TwmdError`
+directly and carry none of those, since no response was received.
+
+| Status | Exception | Retried |
+| --- | --- | --- |
+| 401 | `TwmdAuthError` | no |
+| 402 | `TwmdPaymentRequired` | no |
+| 404 | `TwmdNotFoundError` | no |
+| 422 | `TwmdValidationError` | no |
+| 429 | `TwmdRateLimitError` | yes |
+| 5xx | `TwmdServerError` | yes |
+| network failure | `TwmdTransportError` | yes |
+
+Retries use exponential backoff with jitter. A `Retry-After` header — in either
+RFC 7231 form, delta-seconds or HTTP-date — overrides that and is honoured
+exactly, with no jitter and no shortening, since retrying sooner than the server
+permitted is worse than waiting too long. If it asks for more than
+`RETRY_AFTER_MAX` (120s) the client stops and raises the server's error rather
+than blocking for minutes.
+
+401 messages are annotated with the key-free status of the dataset and ticker
+you asked for, so "I forgot my key" is distinguishable from "that ticker is not
+in the sample set".
+
+### On 402
+
+**No 402 response has been observed from this API.** The probe on 2026-07-21
+found zero endpoints declaring 402 in the published OpenAPI document, and
+unauthenticated requests to key-gated datasets return 401 `missing_api_key`.
+`TwmdPaymentRequired` exists as a documented extension point: if the API does
+return 402 — the untested path is a *valid but unentitled* key, which requires
+live credentials to exercise — the full decoded body is preserved verbatim on
+`.body` and reaches your code intact. The client deliberately does not model or
+require any field layout inside it, since no 402 body exists to model.
+
+```python
+except TwmdPaymentRequired as exc:
+    handle_payment_required(exc.body)   # whole body, nothing dropped
+```
+
+## Source attribution
+
+Pass `source` to mark every request with the integration that produced it, so
+the publisher can attribute traffic:
+
+```python
+client = Client(source="ecosys/tradingagents")
+```
+
+It is attached as a `source` query parameter on every request. A per-call
+`source=` on `get_dataset` overrides the client-wide value. It is an ordinary
+query parameter — it does not change the response, does not enter a data
+response's `request_context.filters`, and carries nothing about the user. Leave
+it unset to send nothing.
+
+## Pagination
+
+```python
+df = client.get_all("twse-daily-price", symbol="2330", limit=1000)
+
+for page in client.iter_pages("twse-daily-price", symbol="2330", limit=500):
+    process(page)
+```
+
+The API publishes no cursor: no `cursor` or `next_cursor` field appears in any
+response or in the OpenAPI document. Paging is `limit`/`offset`, and `offset`
+was measured to be **ignored** on the key-free endpoints — `offset=3` returned
+the same records as `offset=0`.
+
+So pagination here is defensive. It advances `offset` as documented, then stops
+when either a page comes back shorter than `limit`, or a page repeats the
+previous page's first record — the signature of a silently ignored `offset`.
+Against endpoints that ignore `offset` this yields exactly one page, which is
+the correct outcome rather than a failure. `max_pages` caps the loop.
+
+## `as_of`
+
+`get_dataset()` accepts an `as_of` argument and forwards it as a query
+parameter. **Its scope is narrow.** As measured on 2026-07-21:
+
+- `as_of` is declared on exactly four endpoints — `income-statement`,
+  `cash-flow-statement`, `balance-sheet`, `financials` — all of which require an
+  API key.
+- On every other endpoint it is not a declared parameter. The API silently
+  ignores unknown query parameters, returning 200 rather than 422, so passing
+  `as_of` there has no observable effect and raises no error.
+- Its behaviour on the four declared endpoints is **unverified** by this
+  project, which has no credentials to exercise them.
+
+Treat `as_of` as forwarded-but-unconfirmed rather than as a general
+point-in-time facility. The `data_as_of` field in responses is a data-freshness
+date and is a different thing.
+
+## Development
+
+```bash
+pip install -e ".[test]"
+pytest -m "not live"    # offline, no network
+pytest -m live          # hits the real API, key-free paths only
+```
+
+The live suite asserts a sample of the key-free matrix — seven dataset/ticker
+pairs — still matches what the API serves, so drift in those surfaces as a test
+failure. It is a tripwire, not full coverage: the five sample tickers are
+verified only against `twse-daily-price`, and three datasets listed as
+key-required are assumed rather than probed (see
+`access.PRESUMED_KEY_REQUIRED_DATASETS`, and `access.provenance()` to tell the
+two apart).
+
+## Scope
+
+This package retrieves data. It does not generate recommendations, forecasts,
+signals, valuations, or opinions about any security, and nothing it returns
+should be read as such. The data is provided for research and educational
+purposes; verify it against the original sources before relying on it. Use of
+the underlying API is governed by TW Market Data's own terms.
+
+## License
+
+MIT

twmarketdata-0.1.0.dist-info/RECORD

@@ -0,0 +1,9 @@
+twmd/__init__.py,sha256=rbJOYxujUF3BPHF6VCKOuHPFTdA4nFpGTjWjk-JIN3A,1438
+twmd/access.py,sha256=GGDYA14DmTLUsJqtwXE4lIBsKUxhDdje61yUknFf-d0,5225
+twmd/client.py,sha256=qn5MxZx27jUHJoIh3n3GBZ1UbcG-i8T0w14pVRBB3E0,16479
+twmd/errors.py,sha256=uFH69pHrOZ4Wxj6dbZTUaq_rFBiQ-LK4_8z4HjVmo7Q,4572
+twmd/frames.py,sha256=9fkPEW8lKM-cNLPno0qfK2VdvpSQttSumLlMMUnpyzc,5377
+twmarketdata-0.1.0.dist-info/METADATA,sha256=sjL5-qZStlzX-PbaEDkq41JocJ_m4AU1HBBYC2osC2Q,12981
+twmarketdata-0.1.0.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
+twmarketdata-0.1.0.dist-info/licenses/LICENSE,sha256=l7ThmYQSxsHt9D0YFvjVa7rSMIANnBxjDPMLhhWUR1M,1071
+twmarketdata-0.1.0.dist-info/RECORD,,

twmarketdata-0.1.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.27.0
+Root-Is-Purelib: true
+Tag: py3-none-any

twmarketdata-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 TW Market Data
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

twmd/__init__.py

@@ -0,0 +1,63 @@
+"""twmd — data-access client for the TW Market Data API.
+
+Fetches published datasets and returns them as pandas DataFrames, for research
+and educational use. This package is a retrieval layer only: it transports
+records as the API sends them and adds no analysis or interpretation.
+"""
+
+from __future__ import annotations
+
+from .access import (
+    KEY_REQUIRED_DATASETS,
+    OPEN_DATASETS,
+    PRESUMED_KEY_REQUIRED_DATASETS,
+    SAMPLE_DATASETS,
+    SAMPLE_TICKERS,
+    access_tier,
+    explain,
+    is_key_free,
+    provenance,
+)
+from .client import API_KEY_ENV, DEFAULT_BASE_URL, Client
+from .errors import (
+    TwmdAPIError,
+    TwmdAuthError,
+    TwmdConfigError,
+    TwmdError,
+    TwmdNotFoundError,
+    TwmdPaymentRequired,
+    TwmdRateLimitError,
+    TwmdServerError,
+    TwmdTransportError,
+    TwmdValidationError,
+)
+from .frames import to_dataframe
+
+__version__ = "0.1.0"
+
+__all__ = [
+    "Client",
+    "API_KEY_ENV",
+    "DEFAULT_BASE_URL",
+    "SAMPLE_TICKERS",
+    "OPEN_DATASETS",
+    "SAMPLE_DATASETS",
+    "KEY_REQUIRED_DATASETS",
+    "PRESUMED_KEY_REQUIRED_DATASETS",
+    "access_tier",
+    "is_key_free",
+    "explain",
+    "provenance",
+    "to_dataframe",
+    "TwmdError",
+    "TwmdConfigError",
+    "TwmdTransportError",
+    "TwmdAPIError",
+    "TwmdAuthError",
+    "TwmdPaymentRequired",
+    "TwmdNotFoundError",
+    "TwmdValidationError",
+    "TwmdRateLimitError",
+    "TwmdServerError",
+    "__version__",
+]

twmd/access.py

@@ -0,0 +1,145 @@
+"""Which dataset/ticker combinations are reachable without an API key.
+
+This module encodes an **empirically measured** matrix, not an assumption. It
+was produced by probing the live API without credentials on 2026-07-21. The
+headline finding: key-free access is scoped per *dataset*, not globally per
+ticker. The same ticker that returns 200 on ``twse-daily-price`` returns 401 on
+``institutional-flow``.
+
+Three access tiers were observed:
+
+``OPEN``
+    Reachable without a key for any ticker.
+``SAMPLE``
+    Reachable without a key only for the five sample tickers in
+    :data:`SAMPLE_TICKERS`. Any other ticker returns 401 ``missing_api_key``.
+``KEY_REQUIRED``
+    Every request needs a key.
+
+Datasets absent from the tables below are treated as ``KEY_REQUIRED``, which is
+the safe default: the client will still send the request, so an unlisted
+dataset that is in fact open keeps working. This module drives *advisory*
+hints, never a client-side block.
+"""
+
+from __future__ import annotations
+
+from typing import FrozenSet, Optional
+
+#: Tickers served without a key on :data:`SAMPLE_DATASETS`. Measured 2026-07-21.
+SAMPLE_TICKERS: FrozenSet[str] = frozenset({"2330", "2317", "2454", "0050", "2603"})
+
+#: Datasets served without a key for any ticker. Measured 2026-07-21.
+OPEN_DATASETS: FrozenSet[str] = frozenset({"security-master", "market-index"})
+
+#: Datasets served without a key only for :data:`SAMPLE_TICKERS`. Measured 2026-07-21.
+SAMPLE_DATASETS: FrozenSet[str] = frozenset(
+    {"twse-daily-price", "tpex-daily-price", "monthly-revenue"}
+)
+
+#: Datasets *measured* returning 401 without a key, including for sample
+#: tickers. Each of these was probed directly on 2026-07-21.
+KEY_REQUIRED_DATASETS: FrozenSet[str] = frozenset(
+    {
+        "institutional-flow",
+        "technical-indicators",
+        "valuation-data",
+        "market-prices",
+        "financial-metrics",
+        "market-snapshot",
+        "market-breadth",
+        "income-statement",
+    }
+)
+
+#: Datasets *believed* key-gated but never probed. They declare ``x-api-key`` in
+#: the OpenAPI document and sit alongside measured-gated siblings, but no
+#: request was issued against them. Kept separate from
+#: :data:`KEY_REQUIRED_DATASETS` so the distinction between measured and assumed
+#: is not lost. They resolve to the same tier; only the provenance differs.
+PRESUMED_KEY_REQUIRED_DATASETS: FrozenSet[str] = frozenset(
+    {
+        "cash-flow-statement",
+        "balance-sheet",
+        "financials",
+    }
+)
+
+OPEN = "open"
+SAMPLE = "sample"
+KEY_REQUIRED = "key_required"
+
+
+def access_tier(dataset: str) -> str:
+    """Return the access tier for ``dataset``.
+
+    Unknown datasets return :data:`KEY_REQUIRED`, the safe default.
+    """
+    key = dataset.strip("/")
+    if key in OPEN_DATASETS:
+        return OPEN
+    if key in SAMPLE_DATASETS:
+        return SAMPLE
+    return KEY_REQUIRED
+
+
+def provenance(dataset: str) -> str:
+    """How the tier for ``dataset`` was established: ``measured`` or ``assumed``.
+
+    Lets a caller — or a future contract re-probe — tell apart what was actually
+    observed from what is only the safe default.
+    """
+    key = dataset.strip("/")
+    measured = OPEN_DATASETS | SAMPLE_DATASETS | KEY_REQUIRED_DATASETS
+    return "measured" if key in measured else "assumed"
+
+
+def is_key_free(dataset: str, ticker: Optional[str] = None) -> bool:
+    """Whether ``dataset`` (optionally for ``ticker``) is reachable without a key.
+
+    Args:
+        dataset: Dataset path segment, e.g. ``"twse-daily-price"``.
+        ticker: Symbol to check. Required to get a meaningful answer for
+            ``SAMPLE``-tier datasets; without it this returns ``False`` for
+            them, since only five specific tickers qualify.
+
+    Returns:
+        ``True`` if an unauthenticated request is expected to succeed.
+
+    Example:
+        >>> is_key_free("twse-daily-price", "2330")
+        True
+        >>> is_key_free("twse-daily-price", "1101")
+        False
+        >>> is_key_free("security-master", "1101")
+        True
+        >>> is_key_free("institutional-flow", "2330")
+        False
+    """
+    tier = access_tier(dataset)
+    if tier == OPEN:
+        return True
+    if tier == SAMPLE:
+        return ticker is not None and str(ticker) in SAMPLE_TICKERS
+    return False
+
+
+def explain(dataset: str, ticker: Optional[str] = None) -> str:
+    """Human-readable reason for the :func:`is_key_free` verdict.
+
+    Used to enrich 401 messages so a caller can tell "I forgot my key" apart
+    from "this dataset is open but that ticker is not in the sample set".
+    """
+    tier = access_tier(dataset)
+    if tier == OPEN:
+        return f"'{dataset}' is served without an API key for any ticker."
+    if tier == SAMPLE:
+        listed = ", ".join(sorted(SAMPLE_TICKERS))
+        if ticker is not None and str(ticker) in SAMPLE_TICKERS:
+            return f"'{dataset}' is served without an API key for ticker {ticker}."
+        return (
+            f"'{dataset}' is served without an API key only for these sample "
+            f"tickers: {listed}."
+            + (f" Ticker {ticker} is not among them." if ticker is not None else "")
+        )
+    return f"'{dataset}' requires an API key for every request."

twmd/client.py

@@ -0,0 +1,422 @@
+"""Synchronous data-access client for the TW Market Data API.
+
+Retrieves published datasets and returns them as pandas DataFrames. This is a
+transport layer: it fetches records and hands them over unmodified. It does not
+compute, score, rank, or interpret anything.
+"""
+
+from __future__ import annotations
+
+import email.utils
+import json
+import os
+from datetime import datetime
+import random
+import time
+from typing import Any, Dict, Iterator, Mapping, Optional, Sequence
+
+import httpx
+import pandas as pd
+
+from . import access
+from .errors import (
+    TwmdAPIError,
+    TwmdAuthError,
+    TwmdConfigError,
+    TwmdRateLimitError,
+    TwmdServerError,
+    TwmdTransportError,
+    error_for_status,
+)
+from .frames import concat_frames, to_dataframe
+
+DEFAULT_BASE_URL = "https://api.twmarketdata.com"
+API_KEY_ENV = "TWMD_API_KEY"
+BASE_URL_ENV = "TWMD_BASE_URL"
+API_KEY_HEADER = "X-API-Key"
+
+DEFAULT_TIMEOUT = 30.0
+DEFAULT_MAX_RETRIES = 3
+DEFAULT_BACKOFF_FACTOR = 0.5
+DEFAULT_BACKOFF_MAX = 20.0
+
+#: Statuses worth retrying. 401/402/404/422 are terminal — retrying cannot help.
+RETRY_STATUSES = frozenset({429, 500, 502, 503, 504})
+
+#: Query parameter names the API uses for a ticker, in the order it accepts them.
+_TICKER_PARAMS = ("symbol", "ticker")
+
+#: Longest wait honoured from a ``Retry-After`` header, in seconds. Beyond this
+#: the client gives up rather than retrying earlier than the server allowed.
+RETRY_AFTER_MAX = 120.0
+
+
+def _fingerprint(page: "pd.DataFrame") -> str:
+    """Stable identity string for a page's first record.
+
+    Used to detect a server that ignored ``offset`` and re-sent page one.
+    Serialised rather than compared as a dict because a partially-null numeric
+    column yields ``NaN``, and ``NaN != NaN`` makes two identical rows compare
+    unequal — which would silently disable the repeat-page guard and let the
+    loop refetch page one up to ``max_pages`` times.
+    """
+    return json.dumps(page.iloc[0].to_dict(), sort_keys=True, default=str)
+
+
+def parse_retry_after(value: Optional[str]) -> Optional[float]:
+    """Parse a ``Retry-After`` header into seconds.
+
+    RFC 7231 permits two forms: delta-seconds, and an HTTP-date. Both are
+    handled; anything unparseable returns ``None`` so the caller falls back to
+    its own backoff.
+    """
+    if not value:
+        return None
+    text = value.strip()
+    try:
+        return max(0.0, float(text))
+    except ValueError:
+        pass
+    try:
+        when = email.utils.parsedate_to_datetime(text)
+    except (TypeError, ValueError):
+        return None
+    if when is None:
+        return None
+    now = datetime.now(when.tzinfo) if when.tzinfo else datetime.now()
+    return max(0.0, (when - now).total_seconds())
+
+
+class Client:
+    """Client for the TW Market Data read API.
+
+    Credentials are read from the ``TWMD_API_KEY`` environment variable by
+    default; a key is never written to disk or logged by this client. Requests
+    without a key still succeed against the key-free datasets described in
+    :mod:`twmd.access`.
+
+    Args:
+        api_key: API key. Defaults to ``os.environ["TWMD_API_KEY"]``. When
+            neither is set the client runs in key-free mode.
+        base_url: API root. Defaults to ``TWMD_BASE_URL`` or
+            ``https://api.twmarketdata.com``.
+        source: Integration marker attached to every request as a ``source``
+            query parameter, so the publisher can attribute traffic to the
+            integration that produced it (e.g. ``"ecosys/tradingagents"``). It is
+            an ordinary query parameter, does not change the response, and
+            carries nothing about the user. A per-call ``source`` passed to
+            :meth:`get_dataset` overrides it. Leave unset to send nothing.
+        timeout: Per-request timeout in seconds.
+        max_retries: Retry attempts for transient failures (429, 5xx, network
+            errors). Terminal statuses are never retried.
+        backoff_factor: Base for exponential backoff, in seconds.
+        transport: Optional httpx transport, for tests.
+
+    Example:
+        Key-free, zero configuration::
+
+            from twmd import Client
+
+            client = Client()
+            df = client.get_dataset("twse-daily-price", symbol="2330", limit=5)
+            print(df[["date", "close"]])
+            print(df.attrs["lineage"])
+    """
+
+    def __init__(
+        self,
+        api_key: Optional[str] = None,
+        *,
+        base_url: Optional[str] = None,
+        source: Optional[str] = None,
+        timeout: float = DEFAULT_TIMEOUT,
+        max_retries: int = DEFAULT_MAX_RETRIES,
+        backoff_factor: float = DEFAULT_BACKOFF_FACTOR,
+        transport: Optional[httpx.BaseTransport] = None,
+    ) -> None:
+        if max_retries < 0:
+            raise TwmdConfigError("max_retries must be >= 0")
+        self._api_key = api_key if api_key is not None else os.environ.get(API_KEY_ENV)
+        if self._api_key is not None:
+            self._api_key = self._api_key.strip() or None
+        self.base_url = (base_url or os.environ.get(BASE_URL_ENV) or DEFAULT_BASE_URL).rstrip("/")
+        self.source = (source or "").strip() or None
+        self.max_retries = max_retries
+        self.backoff_factor = backoff_factor
+
+        headers = {"Accept": "application/json", "User-Agent": "twmd-python/0.1.0"}
+        if self._api_key:
+            headers[API_KEY_HEADER] = self._api_key
+        self._http = httpx.Client(
+            base_url=self.base_url,
+            headers=headers,
+            timeout=timeout,
+            transport=transport,
+            follow_redirects=True,
+        )
+
+    # -- lifecycle ---------------------------------------------------------
+
+    @property
+    def has_api_key(self) -> bool:
+        """Whether this client is carrying a key. Never exposes the key itself."""
+        return self._api_key is not None
+
+    def close(self) -> None:
+        """Close the underlying connection pool."""
+        self._http.close()
+
+    def __enter__(self) -> "Client":
+        return self
+
+    def __exit__(self, *exc_info: Any) -> None:
+        self.close()
+
+    # -- public API --------------------------------------------------------
+
+    def get_dataset(
+        self,
+        dataset: str,
+        *,
+        as_of: Optional[str] = None,
+        **params: Any,
+    ) -> pd.DataFrame:
+        """Fetch one page of a dataset as a DataFrame.
+
+        Args:
+            dataset: Dataset path segment, e.g. ``"twse-daily-price"``. A full
+                path such as ``"/v2/datasets/twse-daily-price"`` also works.
+            as_of: Point-in-time parameter, forwarded as the ``as_of`` query
+                parameter. **Scope is narrow — read this before relying on it.**
+                As of the contract probe on 2026-07-21, ``as_of`` is declared on
+                exactly four endpoints, all of which require a key:
+                ``income-statement``, ``cash-flow-statement``, ``balance-sheet``
+                and ``financials``. On every other endpoint it is not a declared
+                parameter, and because the API silently ignores unknown query
+                parameters — returning 200 rather than 422 — passing it there
+                has no observable effect and produces no error. Its behaviour on
+                the four declared endpoints has not been verified by this
+                client, which has no credentials to exercise them. Treat it as
+                forwarded-but-unconfirmed. Note that the ``data_as_of`` field in
+                responses is a data-freshness date and is a different thing.
+            **params: Query parameters passed through to the API verbatim,
+                e.g. ``symbol``, ``limit``, ``date_from``, ``date_to``.
+
+        Returns:
+            DataFrame of records. Envelope metadata — ``dataset``, ``count``,
+            ``data_as_of``, ``source_role``, ``lineage``, ``meta`` — is on
+            ``df.attrs``. The API's own ``lineage.not_investment_advice`` flag
+            is preserved as sent.
+
+        Raises:
+            TwmdAuthError: 401. Message is annotated with the key-free status of
+                this dataset/ticker pair.
+            TwmdPaymentRequired: 402, if the API ever returns it. Full body on
+                ``.body``.
+            TwmdAPIError: Any other non-2xx response.
+            TwmdTransportError: Network failure after retries.
+        """
+        query = dict(params)
+        if as_of is not None:
+            query["as_of"] = as_of
+        payload = self._request(self._dataset_path(dataset), query)
+        return to_dataframe(payload)
+
+    def iter_pages(
+        self,
+        dataset: str,
+        *,
+        limit: int = 1000,
+        max_pages: int = 100,
+        as_of: Optional[str] = None,
+        **params: Any,
+    ) -> Iterator[pd.DataFrame]:
+        """Yield successive pages of a dataset.
+
+        **Pagination is defensive by necessity.** The API publishes no cursor:
+        the probe on 2026-07-21 found no ``cursor`` or ``next_cursor`` field in
+        any response and none in the OpenAPI document. Paging is expressed as
+        ``limit``/``offset``, and ``offset`` was measured to be *ignored* on the
+        key-free endpoints — ``offset=3`` returned the same records as
+        ``offset=0``. So this method advances ``offset`` as the API documents,
+        but guards against it having no effect:
+
+        1. Stop when a page returns fewer records than ``limit`` (the last page).
+        2. Stop when a page repeats the previous page's first record, which is
+           what a silently-ignored ``offset`` produces. Without this guard the
+           loop would refetch page one until ``max_pages``.
+
+        In practice, against endpoints that ignore ``offset``, this yields
+        exactly one page. That is the correct outcome, not a failure.
+
+        Args:
+            dataset: Dataset path segment.
+            limit: Records requested per page.
+            max_pages: Hard ceiling on pages fetched, as a runaway guard.
+            as_of: See :meth:`get_dataset`.
+            **params: Additional query parameters.
+
+        Yields:
+            One DataFrame per page.
+        """
+        offset = 0
+        previous_first: Optional[str] = None
+        for _ in range(max_pages):
+            page = self.get_dataset(dataset, as_of=as_of, limit=limit, offset=offset, **params)
+            if page.empty:
+                return
+            first = _fingerprint(page)
+            if previous_first is not None and first == previous_first:
+                # offset had no effect; the server re-sent the same page.
+                return
+            yield page
+            if len(page) < limit:
+                return
+            previous_first = first
+            offset += limit
+
+    def get_all(
+        self,
+        dataset: str,
+        *,
+        limit: int = 1000,
+        max_pages: int = 100,
+        as_of: Optional[str] = None,
+        **params: Any,
+    ) -> pd.DataFrame:
+        """Fetch every page and concatenate. See :meth:`iter_pages` for limits."""
+        pages = list(
+            self.iter_pages(
+                dataset, limit=limit, max_pages=max_pages, as_of=as_of, **params
+            )
+        )
+        return concat_frames(pages)
+
+    def list_datasets(self) -> pd.DataFrame:
+        """Fetch the dataset catalog."""
+        payload = self._request("/v2/datasets", {})
+        return to_dataframe(payload)
+
+    def is_key_free(self, dataset: str, ticker: Optional[str] = None) -> bool:
+        """Whether a dataset/ticker pair is reachable without a key.
+
+        Thin re-export of :func:`twmd.access.is_key_free` so callers can check
+        before requesting.
+        """
+        return access.is_key_free(dataset, ticker)
+
+    # -- internals ---------------------------------------------------------
+
+    @staticmethod
+    def _dataset_path(dataset: str) -> str:
+        name = dataset.strip("/")
+        if name.startswith("v2/"):
+            return "/" + name
+        return "/v2/datasets/" + name
+
+    def _request(self, path: str, params: Mapping[str, Any]) -> Dict[str, Any]:
+        """Issue a GET with retry/backoff and map failures onto exceptions."""
+        clean = {k: v for k, v in params.items() if v is not None}
+        # Attach the integration source marker to every request. An explicit
+        # per-call ``source`` wins, so a caller can override it; otherwise the
+        # client-wide marker is used. It is an ordinary query parameter the API
+        # ignores for data selection (verified: it does not enter the
+        # ``request_context.filters`` of data-shaped responses).
+        if self.source is not None and "source" not in clean:
+            clean["source"] = self.source
+        last_exc: Optional[Exception] = None
+
+        for attempt in range(self.max_retries + 1):
+            try:
+                response = self._http.get(path, params=clean)
+            except httpx.HTTPError as exc:
+                last_exc = TwmdTransportError(f"request to {path} failed: {exc}")
+                if attempt < self.max_retries:
+                    self._sleep_before_retry(attempt, None)
+                    continue
+                raise last_exc from exc
+
+            if response.status_code < 300:
+                return self._decode(response)
+
+            body = self._safe_json(response)
+            exc_obj = error_for_status(
+                response.status_code,
+                body=body,
+                text=response.text,
+                request_url=str(response.request.url),
+            )
+
+            if response.status_code in RETRY_STATUSES and attempt < self.max_retries:
+                if self._sleep_before_retry(attempt, response.headers.get("Retry-After")):
+                    last_exc = exc_obj
+                    continue
+                # Server asked us to wait longer than we are willing to.
+                raise exc_obj
+
+            if isinstance(exc_obj, TwmdAuthError):
+                raise self._annotate_auth_error(exc_obj, path, clean)
+            raise exc_obj
+
+        # Retries exhausted on a retryable status.
+        assert last_exc is not None
+        raise last_exc
+
+    def _decode(self, response: httpx.Response) -> Dict[str, Any]:
+        try:
+            payload = response.json()
+        except ValueError as exc:
+            raise TwmdAPIError(
+                response.status_code,
+                text=response.text,
+                request_url=str(response.request.url),
+            ) from exc
+        if not isinstance(payload, dict):
+            return {"rows": payload if isinstance(payload, list) else [payload]}
+        return payload
+
+    @staticmethod
+    def _safe_json(response: httpx.Response) -> Optional[Dict[str, Any]]:
+        try:
+            payload = response.json()
+        except ValueError:
+            return None
+        return payload if isinstance(payload, dict) else {"detail": payload}
+
+    def _annotate_auth_error(
+        self, exc: TwmdAuthError, path: str, params: Mapping[str, Any]
+    ) -> TwmdAuthError:
+        """Append key-free context to a 401 so the cause is obvious."""
+        dataset = path.rsplit("/", 1)[-1]
+        ticker = next(
+            (str(params[p]) for p in _TICKER_PARAMS if params.get(p) is not None), None
+        )
+        hint = access.explain(dataset, ticker)
+        if not self.has_api_key:
+            hint += f" No API key is set; export {API_KEY_ENV} to send one."
+        exc.args = (f"{exc.args[0]} — {hint}",)
+        return exc
+
+    def _sleep_before_retry(self, attempt: int, retry_after: Optional[str]) -> bool:
+        """Wait before the next attempt. Returns whether a retry should proceed.
+
+        A ``Retry-After`` header is honoured **exactly** — no jitter, no
+        shortening — because retrying sooner than the server permitted is worse
+        than waiting too long. If it asks for longer than
+        :data:`RETRY_AFTER_MAX`, this returns ``False`` so the caller raises the
+        server's own error rather than sleeping for minutes or retrying early.
+
+        Otherwise: exponential backoff with jitter.
+        """
+        delay = parse_retry_after(retry_after)
+        if delay is not None:
+            if delay > RETRY_AFTER_MAX:
+                return False
+            time.sleep(delay)
+            return True
+        backoff = min(self.backoff_factor * (2 ** attempt), DEFAULT_BACKOFF_MAX)
+        time.sleep(backoff * (0.5 + random.random() / 2))
+        return True
+
+
+__all__: Sequence[str] = ["Client"]

twmd/errors.py

@@ -0,0 +1,137 @@
+"""Exception hierarchy for the TWMD data-access client.
+
+Every exception carries the raw decoded response body verbatim on ``.body`` so
+callers can render whatever the API returned without this layer editing,
+summarising, or dropping fields.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Mapping, Optional
+
+
+class TwmdError(Exception):
+    """Base class for every error raised by this client."""
+
+
+class TwmdConfigError(TwmdError):
+    """Client was constructed with an unusable configuration."""
+
+
+class TwmdTransportError(TwmdError):
+    """Network-level failure: DNS, connection, timeout, TLS."""
+
+
+class TwmdAPIError(TwmdError):
+    """The API returned a non-2xx response.
+
+    Attributes:
+        status_code: HTTP status code.
+        body: Decoded response body, verbatim. ``None`` if the body was not
+            valid JSON, in which case ``text`` holds the raw payload.
+        text: Raw response text.
+        error_code: The API's ``error`` field, when present.
+        request_url: URL that produced this response.
+    """
+
+    def __init__(
+        self,
+        status_code: int,
+        *,
+        body: Optional[Mapping[str, Any]] = None,
+        text: str = "",
+        request_url: str = "",
+    ) -> None:
+        self.status_code = status_code
+        self.body = body
+        self.text = text
+        self.request_url = request_url
+        self.error_code = body.get("error") if isinstance(body, Mapping) else None
+        message = ""
+        if isinstance(body, Mapping):
+            message = str(body.get("message") or body.get("detail") or "")
+        if not message:
+            message = text[:200]
+        super().__init__(f"HTTP {status_code}: {message}" if message else f"HTTP {status_code}")
+
+
+class TwmdAuthError(TwmdAPIError):
+    """HTTP 401. No API key was sent, or the key was rejected.
+
+    Observed ``error`` codes:
+
+    - ``missing_api_key`` — the request carried no key and the requested
+      dataset/ticker combination is not in the key-free set. See
+      :mod:`twmd.access`.
+    - ``invalid_api_key`` — a key was sent but is unknown or revoked.
+
+    As of the contract probe on 2026-07-21 this is the status returned for
+    *unauthenticated* access to key-gated datasets. It is not a billing signal.
+    """
+
+
+class TwmdPaymentRequired(TwmdAPIError):
+    """HTTP 402, reserved for a billing/entitlement signal.
+
+    **Not observed in the live contract.** A probe of the published OpenAPI
+    document on 2026-07-21 found zero endpoints declaring a 402 response, and
+    unauthenticated requests to key-gated datasets return
+    :class:`TwmdAuthError` (401) instead. This class exists so that if the API
+    does return 402 — the untested path is a *valid but unentitled* key, which
+    cannot be exercised without live credentials — the response reaches the
+    caller intact rather than being flattened into a generic error.
+
+    The entire decoded body is preserved verbatim on ``.body``. This client
+    deliberately does not model, validate, or require any particular field
+    layout inside it, because no 402 body has been observed to model.
+
+    Example:
+        >>> try:
+        ...     client.get_dataset("institutional-flow", symbol="2330")
+        ... except TwmdPaymentRequired as exc:
+        ...     handle_payment_required(exc.body)  # full body, unmodified
+    """
+
+
+class TwmdNotFoundError(TwmdAPIError):
+    """HTTP 404. Unknown dataset path or resource."""
+
+
+class TwmdValidationError(TwmdAPIError):
+    """HTTP 422. The API rejected a parameter value.
+
+    Note that the API tolerates *unknown* query parameters silently — it
+    returns 200 and ignores them rather than raising 422. See
+    :meth:`twmd.client.Client.get_dataset` for what that means for ``as_of``.
+    """
+
+
+class TwmdRateLimitError(TwmdAPIError):
+    """HTTP 429. Retried automatically, honouring ``Retry-After`` when sent."""
+
+
+class TwmdServerError(TwmdAPIError):
+    """HTTP 5xx. Retried automatically."""
+
+
+_STATUS_MAP = {
+    401: TwmdAuthError,
+    402: TwmdPaymentRequired,
+    404: TwmdNotFoundError,
+    422: TwmdValidationError,
+    429: TwmdRateLimitError,
+}
+
+
+def error_for_status(
+    status_code: int,
+    *,
+    body: Optional[Mapping[str, Any]] = None,
+    text: str = "",
+    request_url: str = "",
+) -> TwmdAPIError:
+    """Map an HTTP status code onto the matching exception class."""
+    cls = _STATUS_MAP.get(status_code)
+    if cls is None:
+        cls = TwmdServerError if status_code >= 500 else TwmdAPIError
+    return cls(status_code, body=body, text=text, request_url=request_url)

twmd/frames.py

@@ -0,0 +1,141 @@
+"""Convert API response envelopes into pandas DataFrames.
+
+The response envelope observed on 2026-07-21 looks like::
+
+    {
+      "dataset": "twse_daily_price",
+      "rows": [ {...}, ... ],
+      "count": 2,
+      "data_as_of": "2026-07-17",
+      "source_role": "official_twse",
+      "lineage": {"provider": "TWSE", ..., "not_investment_advice": true},
+      "meta": {"last_trading_day": "...", "market_status": [...]}
+    }
+
+Not every dataset uses that shape. Two envelope variants were observed on
+2026-07-21:
+
+``rows`` / ``count``
+    Used by ``twse-daily-price``, ``tpex-daily-price``, ``monthly-revenue``.
+    Records are flat. ``monthly-revenue`` sends a shorter envelope carrying only
+    ``dataset``, ``rows`` and ``count``.
+``items`` / ``row_count``
+    Used by ``security-master`` and ``market-index``. Records contain *nested*
+    objects — ``security_identity``, ``market_identity``, ``index_level`` and so
+    on — and the envelope carries extra integrity metadata such as
+    ``survivorship_bias_warning``.
+``data`` / ``quality.row_count``
+    Used by ``delisting`` and ``stock-delisting-lifecycle``. Adds
+    ``request_context`` (the filters the server actually applied, plus the
+    min/max dates it holds), a ``lineage`` block naming the official source
+    family, and a top-level ``error`` slot.
+
+Catalog endpoints under ``/v2/data-catalog/`` name their list after themselves
+(``reconciliation``, ``stats``); these are picked up by a generic fallback
+rather than an entry per endpoint.
+
+:func:`to_dataframe` accepts both. Nested record fields are left as dict-valued
+columns rather than flattened, so what you get back is what the API sent; use
+:func:`pandas.json_normalize` on the column if you want it expanded.
+
+Everything outside the record list is preserved on ``DataFrame.attrs`` rather
+than discarded — including the API's own ``lineage.not_investment_advice`` flag
+and any ``survivorship_bias_warning``, both passed through untouched.
+"""
+
+from __future__ import annotations
+
+from typing import Any, Mapping, Optional, Tuple
+
+import pandas as pd
+
+#: Envelope keys known to hold the record list, in precedence order.
+RECORD_KEYS: Tuple[str, ...] = ("rows", "items", "data")
+
+#: Envelope keys that may hold the server-reported record count.
+COUNT_KEYS: Tuple[str, ...] = ("count", "row_count")
+
+#: Envelope keys that never hold records, so the generic fallback skips them.
+_NON_RECORD_KEYS = frozenset({"lineage", "meta", "quality", "request_context", "error"})
+
+
+def record_key(payload: Mapping[str, Any]) -> Optional[str]:
+    """Return which envelope key holds the records, or ``None`` if none does.
+
+    Tries the known keys first, then falls back to the first list-valued key
+    that is not known metadata. The fallback covers catalog endpoints that name
+    their list after themselves — ``/v2/data-catalog/reconciliation`` returns
+    ``reconciliation``, ``/v2/data-catalog/stats`` returns ``stats`` — without
+    needing an entry per endpoint.
+    """
+    for key in RECORD_KEYS:
+        if isinstance(payload.get(key), list):
+            return key
+    for key, value in payload.items():
+        if key not in _NON_RECORD_KEYS and isinstance(value, list):
+            return key
+    return None
+
+
+def server_count(payload: Mapping[str, Any]) -> Optional[int]:
+    """Return the server-reported record count across every envelope variant.
+
+    Checks the top level first, then ``quality.row_count``, where the
+    ``data``-shaped envelopes put it.
+    """
+    for key in COUNT_KEYS:
+        value = payload.get(key)
+        if isinstance(value, int):
+            return value
+    quality = payload.get("quality")
+    if isinstance(quality, Mapping):
+        for key in COUNT_KEYS:
+            value = quality.get(key)
+            if isinstance(value, int):
+                return value
+    return None
+
+
+def to_dataframe(payload: Mapping[str, Any]) -> pd.DataFrame:
+    """Build a DataFrame from one response envelope.
+
+    Handles both the ``rows``/``count`` and ``items``/``row_count`` variants.
+    Records become rows; every other envelope key is copied onto ``df.attrs``
+    verbatim, so lineage, freshness and integrity metadata survive.
+
+    Args:
+        payload: Decoded JSON response body.
+
+    Returns:
+        A DataFrame of the records. Empty (no columns) when the record list is
+        empty or absent.
+    """
+    key = record_key(payload)
+    records = payload.get(key) or [] if key else []
+    frame = pd.DataFrame(records)
+    frame.attrs.update({k: v for k, v in payload.items() if k != key})
+    return frame
+
+
+def concat_frames(frames: "list[pd.DataFrame]") -> pd.DataFrame:
+    """Concatenate paginated frames, keeping the first page's metadata.
+
+    Args:
+        frames: Frames in page order.
+
+    Returns:
+        One DataFrame with a fresh RangeIndex. ``attrs`` is taken from the
+        first page, with whichever count key that page used — ``count`` or
+        ``row_count`` — replaced by the combined record total.
+    """
+    if not frames:
+        return pd.DataFrame()
+    if len(frames) == 1:
+        return frames[0]
+    combined = pd.concat(frames, ignore_index=True)
+    combined.attrs.update(frames[0].attrs)
+    for key in COUNT_KEYS:
+        if key in combined.attrs:
+            combined.attrs[key] = len(combined)
+    combined.attrs["pages_fetched"] = len(frames)
+    return combined