quantpad-data 0.1.0__tar.gz → 0.2.0__tar.gz

quantpad_data-0.2.0/PKG-INFO

@@ -0,0 +1,159 @@
+Metadata-Version: 2.5
+Name: quantpad-data
+Version: 0.2.0
+Summary: Official Python client for the QuantPad market-data API
+Project-URL: Documentation, https://api.quantpad.ai/external/docs
+Author: QuantPad
+License: Proprietary
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: pandas<3,>=2.2
+Requires-Dist: pyarrow>=18
+Requires-Dist: requests>=2.32
+Provides-Extra: rth
+Requires-Dist: exchange-calendars>=4.5; extra == 'rth'
+Provides-Extra: test
+Requires-Dist: build>=1.2; extra == 'test'
+Requires-Dist: pytest>=8; extra == 'test'
+Description-Content-Type: text/markdown
+
+# QuantPad Data Python SDK
+
+Install into the project's existing environment/package manager:
+
+```bash
+uv add quantpad-data       # uv project
+poetry add quantpad-data   # Poetry project
+```
+
+For a plain Python project, create a local virtual environment:
+
+```bash
+python3 -m venv .venv
+source .venv/bin/activate  # Windows PowerShell: .venv\Scripts\Activate.ps1
+python -m pip install --upgrade pip
+python -m pip install quantpad-data
+export QUANTPAD_API_KEY=qp_live_...
+```
+
+Do not install into Homebrew/system Python or use `--break-system-packages`.
+When a project environment already exists, use its package manager or
+`python -m pip` rather than creating a second environment. Keep
+`QUANTPAD_API_KEY` in the local environment—never source code or chat.
+
+```python
+import time
+import quantpad_data as qpd
+
+end = int(time.time() * 1000)
+bars = qpd.get_bars("ES.FUT", "1m", end - 86_400_000, end)
+
+for chunk in qpd.get_ticks(
+    "AAPL", "trades", end - 3_600_000, end, columns=["t", "price", "size"]
+):
+    print(chunk.head())
+
+# 10-level L2 market-by-price order-book depth. This streams chunks;
+# project columns to avoid materializing all 60+ book fields when unnecessary.
+for chunk in qpd.get_mbp10(
+    "ES.FUT",
+    end - 3_600_000,
+    end,
+    columns=["t", "bid_px_00", "ask_px_00", "bid_sz_00", "ask_sz_00"],
+):
+    print(chunk.head())
+
+matches = qpd.get_universe("apple", asset_class="equity")
+coverage = qpd.get_coverage("AAPL")
+```
+
+`QuantPadClient` (also available as `Client`) accepts `api_key=`, `base_url=`,
+and `max_retries=`. The default
+client honors `Retry-After` and uses exponential backoff for transient failures.
+QuantPad notebooks remain compatible through `QUANTPAD_NOTEBOOK_DATA_TOKEN`.
+
+Bars preserve the notebook helper's OHLCV aliases and smart futures
+back-adjustment default. Tick timestamps are epoch nanoseconds. Install
+`quantpad-data[rth]` for XNYS regular-session helpers.
+
+`mbp-10` is true L2 depth with bid/ask price, size, and order-count fields at
+levels 0-9. It is available for supported CME futures, individual CME
+options-on-futures contracts, and US equities, subject to `get_coverage()` and
+the plan lookback window (currently 30 days).
+
+This SDK intentionally exposes only bars, ticks, universe, coverage, and
+symbology. It does not proxy FRED or SEC EDGAR.
+
+## Explicit addressing: `qpd.v2`
+
+The helpers above infer which dataset a symbol lives in and what kind of
+symbol it is. That is convenient until it is wrong, and when it is wrong it
+fails quietly — asking for a specific futures contract returns an empty frame
+that looks exactly like a coverage gap.
+
+`qpd.v2.Historical` removes the inference. It takes the same arguments as
+`databento.Historical`, so code written against Databento ports over by
+swapping the client:
+
+```python
+import quantpad_data as qpd
+
+client = qpd.v2.Historical()
+
+bars = client.timeseries.get_range(
+    dataset="GLBX.MDP3",       # required — never guessed
+    symbols="ESZ6",            # a specific contract month
+    stype_in="raw_symbol",     # raw_symbol | instrument_id | parent | continuous
+    schema="ohlcv-1m",
+    start="2026-01-05",
+    end="2026-01-06",
+)
+
+# What actually answered, so you can confirm it was the contract you meant.
+print(bars.attrs["quantpad"])
+# {'dataset': 'GLBX.MDP3', 'stype_in': 'raw_symbol',
+#  'resolved': 'ESZ6=651968', 'clamped': False, ...}
+```
+
+Every response reports the address it used, and a window that had to be
+shortened comes back with `clamped=True` and a warning rather than quietly
+covering a different period than you asked for.
+
+### Large ranges
+
+`get_range` buffers a single response. When that would be too large, it falls
+back automatically to reading parquet straight from object storage — the bytes
+never traverse the data service. Force either behaviour with
+`mode="interactive"` or `mode="bulk"`, or stream day by day:
+
+```python
+for day in client.timeseries.iter_range(
+    dataset="GLBX.MDP3", symbols="ESZ6", schema="mbp-10",
+    start="2026-01-05", end="2026-02-05",
+):
+    process(day)
+```
+
+For ranges too large even for that, submit a batch job, then hydrate it into
+the cache so ordinary `get_range` calls serve it from storage:
+
+```python
+job = client.batch.submit(
+    dataset="GLBX.MDP3", symbols="ESZ6", schema="mbo",
+    start="2026-01-05", end="2026-02-05", hydrate=True,
+)
+client.batch.hydrate(job["job_id"])
+```
+
+### Other namespaces
+
+`client.symbology.resolve(...)`, the `client.metadata.*` mirrors
+(`list_datasets`, `list_schemas`, `list_fields`, `list_publishers`,
+`get_dataset_range`, `get_dataset_condition`, `get_record_count`), and
+`client.options.chain(...)` for enumerating strikes and expiries without
+downloading a day's entire `definition` schema.
+
+There is no cost or unit-price surface. QuantPad data is included in the plan
+rather than metered per request.

quantpad_data-0.2.0/README.md

@@ -0,0 +1,138 @@
+# QuantPad Data Python SDK
+
+Install into the project's existing environment/package manager:
+
+```bash
+uv add quantpad-data       # uv project
+poetry add quantpad-data   # Poetry project
+```
+
+For a plain Python project, create a local virtual environment:
+
+```bash
+python3 -m venv .venv
+source .venv/bin/activate  # Windows PowerShell: .venv\Scripts\Activate.ps1
+python -m pip install --upgrade pip
+python -m pip install quantpad-data
+export QUANTPAD_API_KEY=qp_live_...
+```
+
+Do not install into Homebrew/system Python or use `--break-system-packages`.
+When a project environment already exists, use its package manager or
+`python -m pip` rather than creating a second environment. Keep
+`QUANTPAD_API_KEY` in the local environment—never source code or chat.
+
+```python
+import time
+import quantpad_data as qpd
+
+end = int(time.time() * 1000)
+bars = qpd.get_bars("ES.FUT", "1m", end - 86_400_000, end)
+
+for chunk in qpd.get_ticks(
+    "AAPL", "trades", end - 3_600_000, end, columns=["t", "price", "size"]
+):
+    print(chunk.head())
+
+# 10-level L2 market-by-price order-book depth. This streams chunks;
+# project columns to avoid materializing all 60+ book fields when unnecessary.
+for chunk in qpd.get_mbp10(
+    "ES.FUT",
+    end - 3_600_000,
+    end,
+    columns=["t", "bid_px_00", "ask_px_00", "bid_sz_00", "ask_sz_00"],
+):
+    print(chunk.head())
+
+matches = qpd.get_universe("apple", asset_class="equity")
+coverage = qpd.get_coverage("AAPL")
+```
+
+`QuantPadClient` (also available as `Client`) accepts `api_key=`, `base_url=`,
+and `max_retries=`. The default
+client honors `Retry-After` and uses exponential backoff for transient failures.
+QuantPad notebooks remain compatible through `QUANTPAD_NOTEBOOK_DATA_TOKEN`.
+
+Bars preserve the notebook helper's OHLCV aliases and smart futures
+back-adjustment default. Tick timestamps are epoch nanoseconds. Install
+`quantpad-data[rth]` for XNYS regular-session helpers.
+
+`mbp-10` is true L2 depth with bid/ask price, size, and order-count fields at
+levels 0-9. It is available for supported CME futures, individual CME
+options-on-futures contracts, and US equities, subject to `get_coverage()` and
+the plan lookback window (currently 30 days).
+
+This SDK intentionally exposes only bars, ticks, universe, coverage, and
+symbology. It does not proxy FRED or SEC EDGAR.
+
+## Explicit addressing: `qpd.v2`
+
+The helpers above infer which dataset a symbol lives in and what kind of
+symbol it is. That is convenient until it is wrong, and when it is wrong it
+fails quietly — asking for a specific futures contract returns an empty frame
+that looks exactly like a coverage gap.
+
+`qpd.v2.Historical` removes the inference. It takes the same arguments as
+`databento.Historical`, so code written against Databento ports over by
+swapping the client:
+
+```python
+import quantpad_data as qpd
+
+client = qpd.v2.Historical()
+
+bars = client.timeseries.get_range(
+    dataset="GLBX.MDP3",       # required — never guessed
+    symbols="ESZ6",            # a specific contract month
+    stype_in="raw_symbol",     # raw_symbol | instrument_id | parent | continuous
+    schema="ohlcv-1m",
+    start="2026-01-05",
+    end="2026-01-06",
+)
+
+# What actually answered, so you can confirm it was the contract you meant.
+print(bars.attrs["quantpad"])
+# {'dataset': 'GLBX.MDP3', 'stype_in': 'raw_symbol',
+#  'resolved': 'ESZ6=651968', 'clamped': False, ...}
+```
+
+Every response reports the address it used, and a window that had to be
+shortened comes back with `clamped=True` and a warning rather than quietly
+covering a different period than you asked for.
+
+### Large ranges
+
+`get_range` buffers a single response. When that would be too large, it falls
+back automatically to reading parquet straight from object storage — the bytes
+never traverse the data service. Force either behaviour with
+`mode="interactive"` or `mode="bulk"`, or stream day by day:
+
+```python
+for day in client.timeseries.iter_range(
+    dataset="GLBX.MDP3", symbols="ESZ6", schema="mbp-10",
+    start="2026-01-05", end="2026-02-05",
+):
+    process(day)
+```
+
+For ranges too large even for that, submit a batch job, then hydrate it into
+the cache so ordinary `get_range` calls serve it from storage:
+
+```python
+job = client.batch.submit(
+    dataset="GLBX.MDP3", symbols="ESZ6", schema="mbo",
+    start="2026-01-05", end="2026-02-05", hydrate=True,
+)
+client.batch.hydrate(job["job_id"])
+```
+
+### Other namespaces
+
+`client.symbology.resolve(...)`, the `client.metadata.*` mirrors
+(`list_datasets`, `list_schemas`, `list_fields`, `list_publishers`,
+`get_dataset_range`, `get_dataset_condition`, `get_record_count`), and
+`client.options.chain(...)` for enumerating strikes and expiries without
+downloading a day's entire `definition` schema.
+
+There is no cost or unit-price surface. QuantPad data is included in the plan
+rather than metered per request.

{quantpad_data-0.1.0 → quantpad_data-0.2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "quantpad-data"
-version = "0.1.0"
+version = "0.2.0"
 description = "Official Python client for the QuantPad market-data API"
 readme = "README.md"
 requires-python = ">=3.10"

{quantpad_data-0.1.0 → quantpad_data-0.2.0}/quantpad_data/__init__.py

@@ -194,6 +194,14 @@ def iter_l1_mbp1_df_chunks(symbol, start_ms, end_ms, chunksize=250_000,
     return iter_mbp1(symbol, start_ms, end_ms, columns=usecols, chunksize=chunksize)
 
 
+from . import lean  # noqa: E402  (submodule: LEAN backtesting data staging)
+from . import v2  # noqa: E402  (submodule: Databento-shaped explicit API)
+
+# Exported at the root as well as on ``v2``: the sandbox module reaches it
+# as ``qpd.Historical()``, and one spelling across both surfaces means
+# notebook code moves between them unchanged.
+Historical = v2.Historical
+
 __all__ = [
     "Client", "QuantPadClient", "QuantPadDataError", "QuantPadDataNotReady", "AuthenticationError",
     "PermissionDeniedError", "RateLimitError", "QuotaExceededError",
@@ -202,5 +210,6 @@ __all__ = [
     "get_trades", "get_mbp1", "get_mbp10", "get_trades_rth", "get_mbp1_rth",
     "get_mbp10_rth", "get_spread_1m_rth", "symbology_resolve", "get_universe",
     "get_coverage", "iter_trades", "iter_mbp1", "iter_l1_df_chunks",
-    "iter_l1_trades_df_chunks", "iter_l1_mbp1_df_chunks", "np", "pd", "pa",
+    "iter_l1_trades_df_chunks", "iter_l1_mbp1_df_chunks", "lean", "v2",
+    "Historical", "np", "pd", "pa",
 ]

{quantpad_data-0.1.0 → quantpad_data-0.2.0}/quantpad_data/client.py

@@ -24,7 +24,7 @@ from .errors import (
     ValidationError,
 )
 
-VERSION = "0.1.0"
+VERSION = "0.2.0"
 USER_AGENT = f"quantpad-data-python/{VERSION}"
 TICK_SCHEMAS = (
     "trades", "mbp-1", "mbp-10", "cmbp-1", "tcbbo", "cbbo-1s",
@@ -88,20 +88,21 @@ class Client:
             retry_after = float(response.headers["Retry-After"])
         except (KeyError, TypeError, ValueError):
             pass
-        if response.status_code == 400:
+        status = response.status_code
+        if status == 400:
             raise ValidationError(message)
-        if response.status_code == 401:
-            raise AuthenticationError(message)
-        if response.status_code == 403:
-            raise PermissionDeniedError(message)
-        if response.status_code == 404:
+        if status == 401:
+            raise AuthenticationError(message, status)
+        if status == 403:
+            raise PermissionDeniedError(message, status)
+        if status == 404:
             raise NotFoundError(message)
-        if response.status_code == 429:
+        if status == 429:
             cls = QuotaExceededError if "quota" in message.lower() else RateLimitError
-            raise cls(message, retry_after)
-        if response.status_code >= 500:
-            raise UpstreamError(message)
-        raise QuantPadDataError(message)
+            raise cls(message, retry_after, status)
+        if status >= 500:
+            raise UpstreamError(message, status)
+        raise QuantPadDataError(message, status)
 
     def _request(self, method: str, path: str, **kwargs) -> requests.Response:
         kwargs.setdefault("timeout", self.timeout)

quantpad_data-0.2.0/quantpad_data/errors.py

@@ -0,0 +1,60 @@
+"""Typed QuantPad Data API exceptions."""
+
+from __future__ import annotations
+
+
+class QuantPadDataError(RuntimeError):
+    """Base error for API and transport failures.
+
+    ``status_code`` is set when the failure came from an HTTP response.
+    Callers that want to branch on a specific status — the bulk-delivery
+    fallback keys on 413 — can read it without re-parsing the message.
+    """
+
+    #: Class-level default so the attribute is always readable, including
+    #: on the subclasses that also inherit ValueError and therefore never
+    #: run this __init__.
+    status_code: int | None = None
+
+    def __init__(self, message: str, status_code: int | None = None):
+        super().__init__(message)
+        self.status_code = status_code
+
+
+class AuthenticationError(QuantPadDataError):
+    pass
+
+
+class PermissionDeniedError(QuantPadDataError):
+    pass
+
+
+class RateLimitError(QuantPadDataError):
+    def __init__(
+        self,
+        message: str,
+        retry_after: float | None = None,
+        status_code: int | None = None,
+    ):
+        super().__init__(message, status_code)
+        self.retry_after = retry_after
+
+
+class QuotaExceededError(RateLimitError):
+    pass
+
+
+class NotFoundError(ValueError, QuantPadDataError):
+    pass
+
+
+class ValidationError(ValueError, QuantPadDataError):
+    pass
+
+
+class UpstreamError(QuantPadDataError):
+    pass
+
+
+class QuantPadDataNotReady(QuantPadDataError):
+    """Backward-compatible error retained for older notebooks."""

quantpad_data-0.2.0/quantpad_data/lean.py

@@ -0,0 +1,153 @@
+"""Stage QuantConnect LEAN futures data into a local data folder.
+
+Flow (run inside the LEAN sandbox before the engine starts):
+
+    from quantpad_data import lean
+    lean.prepare_and_wait(["MNQ.FUT"], "2025-01-06", "2025-01-31", "minute")
+    lean.stage_data(["MNQ.FUT"], "2025-01-06", "2025-01-31", "minute",
+                    data_folder="/home/jovyan/lean-data")
+
+``prepare`` asks the data service to convert/cache the range (202 + job);
+``stage_data`` fetches the manifest (authenticated) and downloads each
+artifact directly from R2 via short-lived presigned URLs — no zip bytes flow
+through the data service.
+"""
+
+from __future__ import annotations
+
+import os
+import time
+from pathlib import Path
+from typing import Sequence
+
+import requests
+
+from .client import Client
+from .errors import QuantPadDataError
+
+
+class LeanStageError(QuantPadDataError):
+    """Raised when preparing or staging LEAN data fails."""
+
+
+def _client(client: Client | None) -> Client:
+    return client if client is not None else Client()
+
+
+def prepare(
+    symbols: Sequence[str],
+    start: str,
+    end: str,
+    resolution: str,
+    *,
+    client: Client | None = None,
+) -> dict:
+    """Kick off conversion; returns the 202 body (``job_id``, ``poll_url``)."""
+    response = _client(client)._request(
+        "POST",
+        "/v1/lean/prepare",
+        json={
+            "symbols": list(symbols),
+            "start": start,
+            "end": end,
+            "resolution": resolution,
+        },
+    )
+    return response.json()
+
+
+def wait_for_job(
+    job_id: str,
+    *,
+    client: Client | None = None,
+    poll_seconds: float = 2.0,
+    timeout_seconds: float = 1800.0,
+) -> dict:
+    """Poll ``/v1/jobs/{job_id}`` until completed; raises on failure/timeout."""
+    bound = _client(client)
+    deadline = time.monotonic() + timeout_seconds
+    while True:
+        status = bound._request("GET", f"/v1/jobs/{job_id}").json()
+        if status.get("status") == "completed":
+            return status
+        if status.get("status") == "failed":
+            raise LeanStageError(
+                f"lean prepare job {job_id} failed: {status.get('error')}"
+            )
+        if time.monotonic() > deadline:
+            raise LeanStageError(f"lean prepare job {job_id} timed out")
+        time.sleep(poll_seconds)
+
+
+def prepare_and_wait(
+    symbols: Sequence[str],
+    start: str,
+    end: str,
+    resolution: str,
+    *,
+    client: Client | None = None,
+    poll_seconds: float = 2.0,
+    timeout_seconds: float = 1800.0,
+) -> dict:
+    accepted = prepare(symbols, start, end, resolution, client=client)
+    return wait_for_job(
+        accepted["job_id"],
+        client=client,
+        poll_seconds=poll_seconds,
+        timeout_seconds=timeout_seconds,
+    )
+
+
+def get_manifest(
+    symbols: Sequence[str],
+    start: str,
+    end: str,
+    resolution: str,
+    *,
+    client: Client | None = None,
+) -> dict:
+    response = _client(client)._request(
+        "GET",
+        "/v1/lean/manifest",
+        params={
+            "symbols": ",".join(symbols),
+            "start": start,
+            "end": end,
+            "resolution": resolution,
+        },
+    )
+    return response.json()
+
+
+def stage_data(
+    symbols: Sequence[str],
+    start: str,
+    end: str,
+    resolution: str,
+    *,
+    data_folder: str | os.PathLike[str],
+    client: Client | None = None,
+    download_session: requests.Session | None = None,
+) -> dict:
+    """Download a prepared range into ``data_folder`` laid out for LEAN.
+
+    Returns the manifest (including ``mapping_modes``, which the run guard
+    validates against the algorithm's requested mapping mode before the
+    engine starts).
+    """
+    manifest = get_manifest(symbols, start, end, resolution, client=client)
+    session = download_session if download_session is not None else requests.Session()
+    root = Path(data_folder)
+    for entry in manifest["files"]:
+        relative = Path(entry["path"])
+        if relative.is_absolute() or ".." in relative.parts:
+            raise LeanStageError(f"unsafe manifest path: {entry['path']}")
+        target = root / relative
+        target.parent.mkdir(parents=True, exist_ok=True)
+        response = session.get(entry["url"], timeout=120)
+        if response.status_code != 200:
+            raise LeanStageError(
+                f"download failed ({response.status_code}) for {entry['path']}"
+            )
+        target.write_bytes(response.content)
+    return manifest