ohlcvault 0.2.0__tar.gz → 0.2.2__tar.gz

{ohlcvault-0.2.0 → ohlcvault-0.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: ohlcvault
-Version: 0.2.0
+Version: 0.2.2
 Summary: Reproducible, checksummed daily OHLCV data for A-share, HK and US markets — static files, content-addressed snapshots, zero API keys.
 Project-URL: Homepage, https://helpquant.com
 Project-URL: Documentation, https://github.com/helpquant/ohlcvault#readme
@@ -132,6 +132,17 @@ A mirror can be an HTTP(S) URL **or a local directory**:
 ov.connect(mirrors=["/path/to/data"])
 ```
 
+The built-in chain has two entries whose **failure domains are disjoint** — a
+commercial CDN and a self-hosted origin:
+
+| Mirror | Role | Independent failure domain |
+|---|---|---|
+| `https://data.ohlcvault.com` | primary | commercial CDN: account suspension, billing, regional outage |
+| `https://data.helpquant.com` | fallback | own server + own DNS |
+
+Override with `OHLCVAULT_MIRRORS=https://a,https://b`, or pass `mirrors=` / set
+`OHLCVAULT_CACHE` to keep everything local.
+
 Mirrors are tried in order; whichever one succeeds is promoted to first place. Every
 file is checked against the `sha256` in the snapshot manifest, and anything that
 fails is **discarded and the next mirror is tried** — bad bytes are never handed to
@@ -144,7 +155,7 @@ the caller.
 | No silently-corrupted data | Every file verified against the snapshot's `sha256` |
 | No silently-changed history | Sealed months are never rewritten |
 | No unverifiable numbers | Missing adjustment factors raise, instead of returning raw prices |
-| No hidden survivorship bias | Delisted stocks are kept in the universe and in the data |
+| No hidden survivorship bias | Delisted stocks are kept in the universe and in the data (`cn`; see coverage table for `hk`/`us` limits) |
 | Byte-for-byte reproduction | Fixed-point integers, `gzip` with `MTIME=0`, no wall-clock timestamps |
 
 **Delisted stocks matter.** If your backtest universe only contains companies that
@@ -159,12 +170,13 @@ Coverage is declared explicitly in `meta/symbols/{market}.json` under `coverage`
 
 | Market | Status | Gaps |
 |---|---|---|
-| `cn` | Daily bars + indices, 2000→present | **No Beijing Stock Exchange** (upstream source doesn't provide it) |
-| `hk` | Symbol list only | No daily bars yet; **list is current listings only** |
-| `us` | Symbol list only | No daily bars yet; **list is current listings only** |
+| `cn` | Daily bars 1990-12→present (A-shares incl. 1,187 delisted) + 461 indices | **No Beijing Stock Exchange** (upstream source doesn't provide it); trade calendar starts 2000-01 (upstream boundary — earlier trading days must be inferred from bar dates); ETF/bond lists included, bars not collected |
+| `hk` | Daily bars 2,806 stocks + HSI index (index history from 2013-08) | **List is current listings only** (survivorship bias, no IPO dates); calendar & index history start 2013-08 (free index source boundary) |
+| `us` | Daily bars 7,513 stocks + SPX index (index history from 2004-01) | **List is current listings only** (survivorship bias, no IPO dates); 5,733 ETFs listed but bars not collected; calendar & index history start 2004-01 (free index source boundary) |
 
 `hk` / `us` symbol lists carry **survivorship bias** (the upstream source only returns
-currently-listed securities) and have no IPO dates. Their daily bars are pending.
+currently-listed securities) and have no IPO dates. `cn` is unaffected: 1,187 delisted
+stocks are in the list and their full history (e.g. `600001.SH` from 1998) is in the data.
 
 An honest data project states its gaps. A dataset that quietly omits them is worse
 than one that is merely incomplete.

{ohlcvault-0.2.0 → ohlcvault-0.2.2}/README.md

@@ -100,6 +100,17 @@ A mirror can be an HTTP(S) URL **or a local directory**:
 ov.connect(mirrors=["/path/to/data"])
 ```
 
+The built-in chain has two entries whose **failure domains are disjoint** — a
+commercial CDN and a self-hosted origin:
+
+| Mirror | Role | Independent failure domain |
+|---|---|---|
+| `https://data.ohlcvault.com` | primary | commercial CDN: account suspension, billing, regional outage |
+| `https://data.helpquant.com` | fallback | own server + own DNS |
+
+Override with `OHLCVAULT_MIRRORS=https://a,https://b`, or pass `mirrors=` / set
+`OHLCVAULT_CACHE` to keep everything local.
+
 Mirrors are tried in order; whichever one succeeds is promoted to first place. Every
 file is checked against the `sha256` in the snapshot manifest, and anything that
 fails is **discarded and the next mirror is tried** — bad bytes are never handed to
@@ -112,7 +123,7 @@ the caller.
 | No silently-corrupted data | Every file verified against the snapshot's `sha256` |
 | No silently-changed history | Sealed months are never rewritten |
 | No unverifiable numbers | Missing adjustment factors raise, instead of returning raw prices |
-| No hidden survivorship bias | Delisted stocks are kept in the universe and in the data |
+| No hidden survivorship bias | Delisted stocks are kept in the universe and in the data (`cn`; see coverage table for `hk`/`us` limits) |
 | Byte-for-byte reproduction | Fixed-point integers, `gzip` with `MTIME=0`, no wall-clock timestamps |
 
 **Delisted stocks matter.** If your backtest universe only contains companies that
@@ -127,12 +138,13 @@ Coverage is declared explicitly in `meta/symbols/{market}.json` under `coverage`
 
 | Market | Status | Gaps |
 |---|---|---|
-| `cn` | Daily bars + indices, 2000→present | **No Beijing Stock Exchange** (upstream source doesn't provide it) |
-| `hk` | Symbol list only | No daily bars yet; **list is current listings only** |
-| `us` | Symbol list only | No daily bars yet; **list is current listings only** |
+| `cn` | Daily bars 1990-12→present (A-shares incl. 1,187 delisted) + 461 indices | **No Beijing Stock Exchange** (upstream source doesn't provide it); trade calendar starts 2000-01 (upstream boundary — earlier trading days must be inferred from bar dates); ETF/bond lists included, bars not collected |
+| `hk` | Daily bars 2,806 stocks + HSI index (index history from 2013-08) | **List is current listings only** (survivorship bias, no IPO dates); calendar & index history start 2013-08 (free index source boundary) |
+| `us` | Daily bars 7,513 stocks + SPX index (index history from 2004-01) | **List is current listings only** (survivorship bias, no IPO dates); 5,733 ETFs listed but bars not collected; calendar & index history start 2004-01 (free index source boundary) |
 
 `hk` / `us` symbol lists carry **survivorship bias** (the upstream source only returns
-currently-listed securities) and have no IPO dates. Their daily bars are pending.
+currently-listed securities) and have no IPO dates. `cn` is unaffected: 1,187 delisted
+stocks are in the list and their full history (e.g. `600001.SH` from 1998) is in the data.
 
 An honest data project states its gaps. A dataset that quietly omits them is worse
 than one that is merely incomplete.

{ohlcvault-0.2.0 → ohlcvault-0.2.2}/ohlcvault/client.py

@@ -71,7 +71,19 @@ class MarketClient:
 
     @staticmethod
     def _label(m: str) -> str:
-        return m.rsplit("/", 2)[-2] if m.startswith("http") else Path(m).name
+        """镜像的短标签，用于 trace 与 `stats["served_by"]`。
+
+        必须对**裸主机 URL** 也给出可区分的名字：默认镜像链全是
+        `https://<host>` 这种无路径形态，而按「最后一个路径段」取名会得到
+        空串 —— 于是两个不同的镜像在统计里混成同一个 key，
+        「到底命中了主源还是备用源」这个问题就再也答不出来了（实测踩到）。
+        """
+        s = str(m).rstrip("/")
+        if "://" in s:
+            rest = s.split("://", 1)[1]
+            host, _, path = rest.partition("/")
+            return f"{host}/{path}" if path else host
+        return Path(s).name
 
     def _fetch_raw(self, m: str, rel: str) -> bytes:
         if m.startswith("http"):

{ohlcvault-0.2.0 → ohlcvault-0.2.2}/ohlcvault/config.py

@@ -5,9 +5,13 @@
 
 | 镜像 | 角色 | 故障域 |
 |---|---|---|
-| Cloudflare R2 + CDN | 主源 | 商业 CDN：账号封禁、账单异常、区域故障 |
-| GitHub raw / jsDelivr | 备用 1 | 微软系托管：与 Cloudflare 完全独立 |
-| 自有服务器 HTTPS | 备用 2 | 自有机器：唯一完全可控的兜底 |
+| `data.ohlcvault.com`（Cloudflare R2 + CDN） | 主源 | 商业 CDN：账号封禁、账单异常、区域故障 |
+| `data.helpquant.com`（自有服务器 HTTPS） | 备用 1 | 自有机器 + 自有 DNS（DNSPod）：唯一完全可控的兜底 |
+
+历史上这里还有第三个源（`cdn.jsdelivr.net/gh/…` 与 `raw.githubusercontent.com/…`），
+指向数据镜像仓库 `helpquant/ohlcvault-data`。该仓库经 GitHub API 查证 **404，从未
+创建**，两个 URL 都是死链，只会让客户端在冷启动时白付一次超时 —— 已移除。
+日后若真建了 GitHub 镜像仓库，再把它们加回来即可（届时故障域才是真正独立的）。
 
 客户端会**记住哪个镜像成功过**并把它提到首位，所以镜像顺序不是性能关键，
 减少的是「每次请求都先付一次失败超时」。
@@ -33,14 +37,13 @@ NAMESPACES = ("stock", "index")
 
 # HTTP User-Agent。必须带自定义 UA：Cloudflare 等边缘节点默认拦截
 # Python-urllib/* 的请求（403），裸 urllib 会被当成 bot。
-VERSION = "0.2.0"
+VERSION = "0.2.2"
 USER_AGENT = f"ohlcvault/{VERSION} (+https://github.com/helpquant/ohlcvault)"
 
 # 默认镜像链。可用 OHLCVAULT_MIRRORS 覆盖。
 DEFAULT_MIRRORS: list[str] = [
     "https://data.ohlcvault.com",
-    "https://cdn.jsdelivr.net/gh/helpquant/ohlcvault-data@main",
-    "https://raw.githubusercontent.com/helpquant/ohlcvault-data/main",
+    "https://data.helpquant.com",
 ]
 
 DEFAULT_CACHE = Path(

{ohlcvault-0.2.0 → ohlcvault-0.2.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "ohlcvault"
-version = "0.2.0"
+version = "0.2.2"
 description = "Reproducible, checksummed daily OHLCV data for A-share, HK and US markets — static files, content-addressed snapshots, zero API keys."
 readme = "README.md"
 requires-python = ">=3.10"

{ohlcvault-0.2.0 → ohlcvault-0.2.2}/tests/conftest.py

@@ -52,9 +52,39 @@ def store(data_dir, tmp_path):
     return ov.connect(mirrors=[str(data_dir)], cache_dir=tmp_path / "cache", quiet=True)
 
 
+def _copy_tree_cow(src: Path, dst: Path) -> None:
+    """复制目录树，尽可能用「写时复制」（APFS clonefile / reflink）。
+
+    mirrors 夹具要为**每个**测试准备多份数据副本；真实数据集在 GB 量级，
+    逐字节复制三遍就是数个 GB —— 会直接吃光开发机磁盘（实测测试进程被
+    SIGKILL 打断，就是因为这里）。COW 副本几乎不额外占空间，而且**写入时
+    自动分离**，所以「篡改某个副本」的测试语义完全成立（硬链接做不到：
+    原地改写会串到其他副本）。平台不支持时回退到普通复制。
+    """
+    import subprocess
+
+    dst.parent.mkdir(parents=True, exist_ok=True)
+    if sys.platform == "darwin":
+        cmd = ["cp", "-Rc"]
+    elif sys.platform.startswith("linux"):
+        cmd = ["cp", "-a", "--reflink=auto"]
+    else:
+        cmd = []
+    if cmd:
+        r = subprocess.run(cmd + [str(src), str(dst)],
+                           stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
+        if r.returncode == 0:
+            return
+    shutil.copytree(src, dst, dirs_exist_ok=True)
+
+
 @pytest.fixture()
 def mirrors(data_dir, tmp_path):
-    """三份独立副本，模拟「R2 / jsDelivr / GitHub raw」的镜像链。
+    """三份独立副本，模拟「主源 + 两个备用源」的镜像链。
+
+    这里的三份是**占位符**（名字沿用历史，不代表当前真实镜像链 —— 真实链见
+    `ohlcvault.config.DEFAULT_MIRRORS`，2026-10-09 起只有两个源）。保留三份是为了
+    让「N 路降级」这条逻辑本身被测到，而不是绑死在当前有几个源上。
 
     用真实字节而不是合成数据 —— 校验、降级、篡改拒绝这些路径只有在
     真实的 gzip 分片上跑才有意义。
@@ -62,8 +92,9 @@ def mirrors(data_dir, tmp_path):
     base = tmp_path / "remote"
     names = ["cloudflare_r2", "jsdelivr", "github_raw"]
     for n in names:
-        shutil.copytree(data_dir, base / n,
-                        ignore=shutil.ignore_patterns(".cache", ".months"))
+        _copy_tree_cow(data_dir, base / n)
+        for skip in (".cache", ".months"):
+            shutil.rmtree(base / n / skip, ignore_errors=True)
     return [str(base / n) for n in names], base