ChatHFD 0.1.2__tar.gz → 0.1.3__tar.gz

chathfd-0.1.3/PKG-INFO

@@ -0,0 +1,76 @@
+Metadata-Version: 2.4
+Name: ChatHFD
+Version: 0.1.3
+Summary: ChatHFD: ChatArch Hugging Face Download tooling package
+Author-email: ChatArch <1073853456@qq.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/ChatArch/ChatHFD
+Project-URL: Repository, https://github.com/ChatArch/ChatHFD
+Project-URL: Documentation, https://arch.gh.wzhecnu.cn/ChatHFD/
+Keywords: chathfd,chatarch,cli
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: click>=8.0
+Requires-Dist: chatstyle<0.3.0,>=0.2.0
+Requires-Dist: chatenv<0.3.0,>=0.2.10
+Provides-Extra: dev
+Requires-Dist: build; extra == "dev"
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: twine; extra == "dev"
+Provides-Extra: docs
+Requires-Dist: mkdocs<2.0,>=1.6; extra == "docs"
+Requires-Dist: mkdocs-material<10.0,>=9.5; extra == "docs"
+Requires-Dist: mkdocs-static-i18n<2.0,>=1.2; extra == "docs"
+Requires-Dist: mike<3.0,>=2.0; extra == "docs"
+Dynamic: license-file
+
+<div align="center">
+    <a href="https://pypi.python.org/pypi/ChatHFD"><img src="https://img.shields.io/pypi/v/ChatHFD.svg" alt="PyPI version" /></a>
+    <a href="https://github.com/ChatArch/ChatHFD/actions/workflows/ci.yml"><img src="https://github.com/ChatArch/ChatHFD/actions/workflows/ci.yml/badge.svg" alt="Tests" /></a>
+</div>
+
+<div align="center">[英文版](README.en.md) | [简体中文](README.md)</div>
+
+# ChatHFD
+
+ChatHFD 是 ChatArch 的 Hugging Face 模型与数据集下载工具。要求 Python 3.10+ 和 `aria2c`；只支持公开仓库，不在命令行传递 Token。
+
+## 下载示例
+
+```bash
+pip install ChatHFD==0.1.3
+chathfd --tree
+chathfd setup  # 检查 aria2c；未安装时提示命令
+# 如确需安装：chathfd setup --install（还需确认）
+# 先预检最新 Qwen 通用开放权重系列（此仓库约 360 GB，不要直接全量下载）
+chathfd download Qwen/Qwen3.8-Flash-Next --include config.json --dry-run
+chathfd download Qwen/Qwen3.8-Flash-Next --include config.json --local-dir ./models/qwen-sample
+# 数学数据集：约 246 KB，包含 README、元数据和一个 Parquet 分片
+chathfd download MathArena/arxivmath-0926 --repo-type dataset --dry-run
+chathfd download MathArena/arxivmath-0926 --repo-type dataset --local-dir ./datasets/arxivmath-0926
+```
+
+默认端点为 `https://hf-mirror.com`；`HF_ENDPOINT` 环境变量或 `--endpoint` 可覆盖为其它 Hugging Face 兼容 HTTPS 端点（如 `https://huggingface.co`）。使用 `--include` / `--exclude`（可重复）筛选仓库路径，`--revision` 固定版本，`--jobs`（默认 5）控制并发文件数，`--connections`（默认 4）控制每文件连接数。下载通过 aria2 续传，完成后只核对文件大小，不做加密哈希校验。若已有文件大小不匹配且无 aria2 续传状态，命令会报错而不会删除它。
+
+默认最多选择 10 GiB 文件；如确需更大的全量权重，先看 `--dry-run` 的大小与磁盘空间，再明确使用 `--allow-large`。元数据响应的 commit SHA 会固定到下载 URL，目标目录也记录仓库、commit 和选中文件集合；再次运行同一集合可续传，分支更新、选中文件集合改变或目标目录已有未知文件时应另选目录。10 GiB 是元数据预检，不是服务器返回字节数的硬上限。`--dry-run` 不创建目标文件，即使超过 10 GiB 仍可预览。默认目录为 `./models/<org>/<repo>` 或 `./datasets/<org>/<repo>`。
+
+ModelScope 是独立仓库平台，不是 HF-Mirror 的 `HF_ENDPOINT` 替代地址；仅当该模型/数据集也在 ModelScope 发布时，另用其 SDK，例如 `modelscope download --model Qwen/Qwen2.5-0.5B-Instruct --local_dir ./model`。不能假定 HF 仓库 ID 在 ModelScope 同步存在，也不能把其 SDK 当作本命令的自动回退。
+
+参考：[HF-Mirror 下载说明](https://hf-mirror.com/) · [ModelScope CLI](https://www.modelscope.cn/docs/sdk/cli) · [Qwen 模型](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) · [MathArena 数据集](https://huggingface.co/datasets/MathArena/arxivmath-0926)。
+
+## 当前 CLI 树（`--tree`）
+
+```text
+chathfd
+├── --help  # Show this message and exit.
+├── --version  # Show the version and exit.
+├── --tree  # Print the registered CLI tree and exit.
+├── --tree-brief  # Print the registered CLI tree without parameter signatures and exit.
+├── download <REPO-ID> [--repo-type REPO-TYPE] [--revision REVISION] [--endpoint ENDPOINT] [--local-dir LOCAL-DIR] [--include INCLUDE] [--exclude EXCLUDE] [--connections CONNECTIONS] [--jobs JOBS] [--dry-run] [--allow-large]  # Download a public model or dataset with aria2c.
+└── setup [--install]  # Check the aria2c download engine and offer installation.
+```
+
+`--tree-brief` 省略子命令参数签名。`src/` 是源码；`tests/cli-tests/` 是真实 CLI 测试；`tests/mock-cli-tests/` 是模拟测试；`docs/` 是长期文档。扩展前请阅读 `DEVELOP.md` 和 `AGENTS.md`。

chathfd-0.1.3/README.md

@@ -0,0 +1,47 @@
+<div align="center">
+    <a href="https://pypi.python.org/pypi/ChatHFD"><img src="https://img.shields.io/pypi/v/ChatHFD.svg" alt="PyPI version" /></a>
+    <a href="https://github.com/ChatArch/ChatHFD/actions/workflows/ci.yml"><img src="https://github.com/ChatArch/ChatHFD/actions/workflows/ci.yml/badge.svg" alt="Tests" /></a>
+</div>
+
+<div align="center">[英文版](README.en.md) | [简体中文](README.md)</div>
+
+# ChatHFD
+
+ChatHFD 是 ChatArch 的 Hugging Face 模型与数据集下载工具。要求 Python 3.10+ 和 `aria2c`；只支持公开仓库，不在命令行传递 Token。
+
+## 下载示例
+
+```bash
+pip install ChatHFD==0.1.3
+chathfd --tree
+chathfd setup  # 检查 aria2c；未安装时提示命令
+# 如确需安装：chathfd setup --install（还需确认）
+# 先预检最新 Qwen 通用开放权重系列（此仓库约 360 GB，不要直接全量下载）
+chathfd download Qwen/Qwen3.8-Flash-Next --include config.json --dry-run
+chathfd download Qwen/Qwen3.8-Flash-Next --include config.json --local-dir ./models/qwen-sample
+# 数学数据集：约 246 KB，包含 README、元数据和一个 Parquet 分片
+chathfd download MathArena/arxivmath-0926 --repo-type dataset --dry-run
+chathfd download MathArena/arxivmath-0926 --repo-type dataset --local-dir ./datasets/arxivmath-0926
+```
+
+默认端点为 `https://hf-mirror.com`；`HF_ENDPOINT` 环境变量或 `--endpoint` 可覆盖为其它 Hugging Face 兼容 HTTPS 端点（如 `https://huggingface.co`）。使用 `--include` / `--exclude`（可重复）筛选仓库路径，`--revision` 固定版本，`--jobs`（默认 5）控制并发文件数，`--connections`（默认 4）控制每文件连接数。下载通过 aria2 续传，完成后只核对文件大小，不做加密哈希校验。若已有文件大小不匹配且无 aria2 续传状态，命令会报错而不会删除它。
+
+默认最多选择 10 GiB 文件；如确需更大的全量权重，先看 `--dry-run` 的大小与磁盘空间，再明确使用 `--allow-large`。元数据响应的 commit SHA 会固定到下载 URL，目标目录也记录仓库、commit 和选中文件集合；再次运行同一集合可续传，分支更新、选中文件集合改变或目标目录已有未知文件时应另选目录。10 GiB 是元数据预检，不是服务器返回字节数的硬上限。`--dry-run` 不创建目标文件，即使超过 10 GiB 仍可预览。默认目录为 `./models/<org>/<repo>` 或 `./datasets/<org>/<repo>`。
+
+ModelScope 是独立仓库平台，不是 HF-Mirror 的 `HF_ENDPOINT` 替代地址；仅当该模型/数据集也在 ModelScope 发布时，另用其 SDK，例如 `modelscope download --model Qwen/Qwen2.5-0.5B-Instruct --local_dir ./model`。不能假定 HF 仓库 ID 在 ModelScope 同步存在，也不能把其 SDK 当作本命令的自动回退。
+
+参考：[HF-Mirror 下载说明](https://hf-mirror.com/) · [ModelScope CLI](https://www.modelscope.cn/docs/sdk/cli) · [Qwen 模型](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) · [MathArena 数据集](https://huggingface.co/datasets/MathArena/arxivmath-0926)。
+
+## 当前 CLI 树（`--tree`）
+
+```text
+chathfd
+├── --help  # Show this message and exit.
+├── --version  # Show the version and exit.
+├── --tree  # Print the registered CLI tree and exit.
+├── --tree-brief  # Print the registered CLI tree without parameter signatures and exit.
+├── download <REPO-ID> [--repo-type REPO-TYPE] [--revision REVISION] [--endpoint ENDPOINT] [--local-dir LOCAL-DIR] [--include INCLUDE] [--exclude EXCLUDE] [--connections CONNECTIONS] [--jobs JOBS] [--dry-run] [--allow-large]  # Download a public model or dataset with aria2c.
+└── setup [--install]  # Check the aria2c download engine and offer installation.
+```
+
+`--tree-brief` 省略子命令参数签名。`src/` 是源码；`tests/cli-tests/` 是真实 CLI 测试；`tests/mock-cli-tests/` 是模拟测试；`docs/` 是长期文档。扩展前请阅读 `DEVELOP.md` 和 `AGENTS.md`。

chathfd-0.1.3/src/ChatHFD.egg-info/PKG-INFO

@@ -0,0 +1,76 @@
+Metadata-Version: 2.4
+Name: ChatHFD
+Version: 0.1.3
+Summary: ChatHFD: ChatArch Hugging Face Download tooling package
+Author-email: ChatArch <1073853456@qq.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/ChatArch/ChatHFD
+Project-URL: Repository, https://github.com/ChatArch/ChatHFD
+Project-URL: Documentation, https://arch.gh.wzhecnu.cn/ChatHFD/
+Keywords: chathfd,chatarch,cli
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: click>=8.0
+Requires-Dist: chatstyle<0.3.0,>=0.2.0
+Requires-Dist: chatenv<0.3.0,>=0.2.10
+Provides-Extra: dev
+Requires-Dist: build; extra == "dev"
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: twine; extra == "dev"
+Provides-Extra: docs
+Requires-Dist: mkdocs<2.0,>=1.6; extra == "docs"
+Requires-Dist: mkdocs-material<10.0,>=9.5; extra == "docs"
+Requires-Dist: mkdocs-static-i18n<2.0,>=1.2; extra == "docs"
+Requires-Dist: mike<3.0,>=2.0; extra == "docs"
+Dynamic: license-file
+
+<div align="center">
+    <a href="https://pypi.python.org/pypi/ChatHFD"><img src="https://img.shields.io/pypi/v/ChatHFD.svg" alt="PyPI version" /></a>
+    <a href="https://github.com/ChatArch/ChatHFD/actions/workflows/ci.yml"><img src="https://github.com/ChatArch/ChatHFD/actions/workflows/ci.yml/badge.svg" alt="Tests" /></a>
+</div>
+
+<div align="center">[英文版](README.en.md) | [简体中文](README.md)</div>
+
+# ChatHFD
+
+ChatHFD 是 ChatArch 的 Hugging Face 模型与数据集下载工具。要求 Python 3.10+ 和 `aria2c`；只支持公开仓库，不在命令行传递 Token。
+
+## 下载示例
+
+```bash
+pip install ChatHFD==0.1.3
+chathfd --tree
+chathfd setup  # 检查 aria2c；未安装时提示命令
+# 如确需安装：chathfd setup --install（还需确认）
+# 先预检最新 Qwen 通用开放权重系列（此仓库约 360 GB，不要直接全量下载）
+chathfd download Qwen/Qwen3.8-Flash-Next --include config.json --dry-run
+chathfd download Qwen/Qwen3.8-Flash-Next --include config.json --local-dir ./models/qwen-sample
+# 数学数据集：约 246 KB，包含 README、元数据和一个 Parquet 分片
+chathfd download MathArena/arxivmath-0926 --repo-type dataset --dry-run
+chathfd download MathArena/arxivmath-0926 --repo-type dataset --local-dir ./datasets/arxivmath-0926
+```
+
+默认端点为 `https://hf-mirror.com`；`HF_ENDPOINT` 环境变量或 `--endpoint` 可覆盖为其它 Hugging Face 兼容 HTTPS 端点（如 `https://huggingface.co`）。使用 `--include` / `--exclude`（可重复）筛选仓库路径，`--revision` 固定版本，`--jobs`（默认 5）控制并发文件数，`--connections`（默认 4）控制每文件连接数。下载通过 aria2 续传，完成后只核对文件大小，不做加密哈希校验。若已有文件大小不匹配且无 aria2 续传状态，命令会报错而不会删除它。
+
+默认最多选择 10 GiB 文件；如确需更大的全量权重，先看 `--dry-run` 的大小与磁盘空间，再明确使用 `--allow-large`。元数据响应的 commit SHA 会固定到下载 URL，目标目录也记录仓库、commit 和选中文件集合；再次运行同一集合可续传，分支更新、选中文件集合改变或目标目录已有未知文件时应另选目录。10 GiB 是元数据预检，不是服务器返回字节数的硬上限。`--dry-run` 不创建目标文件，即使超过 10 GiB 仍可预览。默认目录为 `./models/<org>/<repo>` 或 `./datasets/<org>/<repo>`。
+
+ModelScope 是独立仓库平台，不是 HF-Mirror 的 `HF_ENDPOINT` 替代地址；仅当该模型/数据集也在 ModelScope 发布时，另用其 SDK，例如 `modelscope download --model Qwen/Qwen2.5-0.5B-Instruct --local_dir ./model`。不能假定 HF 仓库 ID 在 ModelScope 同步存在，也不能把其 SDK 当作本命令的自动回退。
+
+参考：[HF-Mirror 下载说明](https://hf-mirror.com/) · [ModelScope CLI](https://www.modelscope.cn/docs/sdk/cli) · [Qwen 模型](https://huggingface.co/Qwen/Qwen3.8-Flash-Next) · [MathArena 数据集](https://huggingface.co/datasets/MathArena/arxivmath-0926)。
+
+## 当前 CLI 树（`--tree`）
+
+```text
+chathfd
+├── --help  # Show this message and exit.
+├── --version  # Show the version and exit.
+├── --tree  # Print the registered CLI tree and exit.
+├── --tree-brief  # Print the registered CLI tree without parameter signatures and exit.
+├── download <REPO-ID> [--repo-type REPO-TYPE] [--revision REVISION] [--endpoint ENDPOINT] [--local-dir LOCAL-DIR] [--include INCLUDE] [--exclude EXCLUDE] [--connections CONNECTIONS] [--jobs JOBS] [--dry-run] [--allow-large]  # Download a public model or dataset with aria2c.
+└── setup [--install]  # Check the aria2c download engine and offer installation.
+```
+
+`--tree-brief` 省略子命令参数签名。`src/` 是源码；`tests/cli-tests/` 是真实 CLI 测试；`tests/mock-cli-tests/` 是模拟测试；`docs/` 是长期文档。扩展前请阅读 `DEVELOP.md` 和 `AGENTS.md`。

{chathfd-0.1.2 → chathfd-0.1.3}/src/ChatHFD.egg-info/SOURCES.txt

@@ -10,6 +10,8 @@ src/ChatHFD.egg-info/top_level.txt
 src/chathfd/__init__.py
 src/chathfd/cli.py
 src/chathfd/config.py
+src/chathfd/download.py
+src/chathfd/setup.py
 tests/test_cli.py
 tests/test_version.py
 tests/test_workflow_contract.py

{chathfd-0.1.2 → chathfd-0.1.3}/src/chathfd/__init__.py

@@ -2,4 +2,4 @@
 
 __all__ = ["__version__"]
 
-__version__ = "0.1.2"
+__version__ = "0.1.3"

chathfd-0.1.3/src/chathfd/cli.py

@@ -0,0 +1,58 @@
+"""CLI entrypoint for chathfd."""
+
+from __future__ import annotations
+
+import os
+from pathlib import Path
+
+import click
+from chatstyle import add_tree_option
+
+from chathfd import __version__
+
+
+@click.group(invoke_without_command=True)
+@click.version_option(__version__, prog_name="chathfd")
+@add_tree_option(renderer_options={"root_name": "chathfd"})
+@click.pass_context
+def main(ctx: click.Context) -> None:
+    """chathfd command line interface."""
+    if ctx.invoked_subcommand is None:
+        # Show help on bare invocation while registered commands stay discoverable.
+        click.echo(ctx.get_help())
+
+
+@main.command()
+@click.argument("repo_id")
+@click.option("--repo-type", type=click.Choice(["model", "dataset"]), default="model", show_default=True)
+@click.option("--revision", default="main", show_default=True, help="Branch, tag or commit.")
+@click.option("--endpoint", default=lambda: os.environ.get("HF_ENDPOINT") or "https://hf-mirror.com", show_default="HF_ENDPOINT or https://hf-mirror.com", help="Hugging Face-compatible HTTPS origin.")
+@click.option("--local-dir", type=click.Path(path_type=Path, file_okay=False), help="Destination (default: ./models or ./datasets / repo ID).")
+@click.option("--include", multiple=True, help="Glob pattern to include (repeatable).")
+@click.option("--exclude", multiple=True, help="Glob pattern to exclude (repeatable).")
+@click.option("--connections", type=click.IntRange(1, 10), default=4, show_default=True, help="Connections per file for aria2c.")
+@click.option("--jobs", type=click.IntRange(1, 10), default=5, show_default=True, help="Concurrent files for aria2c.")
+@click.option("--dry-run", is_flag=True, help="List files and sizes without downloading.")
+@click.option("--allow-large", is_flag=True, help="Allow a selection above 10 GiB.")
+def download(
+    repo_id: str, repo_type: str, revision: str, endpoint: str, local_dir: Path | None,
+    include: tuple[str, ...], exclude: tuple[str, ...], connections: int, jobs: int,
+    dry_run: bool, allow_large: bool,
+) -> None:
+    """Download a public model or dataset with aria2c."""
+    from chathfd.download import download_repo
+
+    download_repo(repo_id, repo_type, revision, endpoint, local_dir, include, exclude, connections, jobs, dry_run, allow_large)
+
+
+@main.command()
+@click.option("--install", is_flag=True, help="Install missing aria2c after confirmation.")
+def setup(install: bool) -> None:
+    """Check the aria2c download engine and offer installation."""
+    from chathfd.setup import setup_aria2
+
+    setup_aria2(install)
+
+
+if __name__ == "__main__":
+    main()

chathfd-0.1.3/src/chathfd/download.py

@@ -0,0 +1,199 @@
+"""Bounded, resumable public Hub downloads through aria2c."""
+
+from __future__ import annotations
+
+import fnmatch
+import json
+import re
+import shutil
+import subprocess
+from pathlib import Path, PurePosixPath
+from urllib.error import HTTPError, URLError
+from urllib.parse import quote, urlsplit
+from urllib.request import urlopen
+
+import click
+
+DEFAULT_ENDPOINT = "https://hf-mirror.com"
+DEFAULT_MAX_BYTES = 10 * 1024**3
+_ID_PART = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$")
+_REV_PART = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$")
+
+
+def _parts(value: str, pattern: re.Pattern[str], label: str) -> list[str]:
+    parts = value.split("/")
+    if not all(part not in (".", "..") and pattern.fullmatch(part) for part in parts):
+        raise click.ClickException(f"Invalid {label}: {value!r}")
+    return parts
+
+
+def _endpoint(value: str) -> str:
+    parsed = urlsplit(value)
+    if parsed.scheme != "https" or not parsed.hostname or parsed.username or parsed.password or parsed.query or parsed.fragment or parsed.path not in ("", "/"):
+        raise click.ClickException("Endpoint must be an HTTPS origin without credentials, path or query")
+    return value.rstrip("/")
+
+
+def fetch_metadata(endpoint: str, repo_id: str, repo_type: str, revision: str) -> dict:
+    prefix = "models" if repo_type == "model" else "datasets"
+    segments = ["api", prefix, *_parts(repo_id, _ID_PART, "repository ID")]
+    if revision != "main":
+        segments += ["revision", *_parts(revision, _REV_PART, "revision")]
+    url = endpoint + "/" + "/".join(quote(part, safe="") for part in segments) + "?blobs=true"
+    try:
+        with urlopen(url, timeout=30) as response:
+            payload = response.read(4 * 1024 * 1024 + 1)
+    except (HTTPError, URLError, TimeoutError) as exc:
+        # Do not include the exception's URL or headers in output.
+        raise click.ClickException(f"Failed to fetch {repo_type} metadata from {endpoint}: {type(exc).__name__}") from exc
+    if len(payload) > 4 * 1024 * 1024:
+        raise click.ClickException("Repository metadata exceeds 4 MiB; refusing incomplete preflight")
+    try:
+        data = json.loads(payload)
+    except (ValueError, UnicodeDecodeError) as exc:
+        raise click.ClickException("Invalid repository metadata response") from exc
+    if not isinstance(data, dict):
+        raise click.ClickException("Unexpected repository metadata response")
+    return data
+
+
+def _selected_files(metadata: dict, includes: tuple[str, ...], excludes: tuple[str, ...]) -> list[tuple[str, int]]:
+    if metadata.get("gated") or metadata.get("private"):
+        raise click.ClickException("Gated/private repositories are not supported; no token is forwarded")
+    siblings = metadata.get("siblings")
+    if not isinstance(siblings, list):
+        raise click.ClickException("Repository metadata has no file list")
+    files = []
+    for entry in siblings:
+        name = entry.get("rfilename") if isinstance(entry, dict) else None
+        if not isinstance(name, str):
+            raise click.ClickException("Repository metadata contains an invalid filename")
+        path = PurePosixPath(name)
+        if path.is_absolute() or "\\" in name or any(ord(char) < 32 or ord(char) == 127 for char in name) or any(part in ("", ".", "..") for part in name.split("/")):
+            raise click.ClickException(f"Unsafe repository path: {name!r}")
+        if includes and not any(fnmatch.fnmatchcase(name, p) for p in includes):
+            continue
+        if any(fnmatch.fnmatchcase(name, p) for p in excludes):
+            continue
+        if path.parts[0] == ".chathfd":
+            raise click.ClickException("Repository file conflicts with the reserved .chathfd directory")
+        size = entry.get("size")
+        if not isinstance(size, int) or isinstance(size, bool) or size < 0:
+            raise click.ClickException(f"Missing file size for {name!r}; cannot preflight")
+        files.append((name, size))
+    if not files:
+        raise click.ClickException("No files match the include/exclude filters")
+    names = {name for name, _ in files}
+    for name in names:
+        if name + ".aria2" in names or any(str(parent) in names for parent in PurePosixPath(name).parents if str(parent) != "."):
+            raise click.ClickException(f"Repository file paths conflict with each other: {name!r}")
+    return files
+
+
+def _check_paths(root: Path, files: list[tuple[str, int]]) -> None:
+    base = root.resolve()
+    if any(ord(char) < 32 or ord(char) == 127 for char in str(base)):
+        raise click.ClickException("Destination contains control characters unsafe for aria2c")
+    for name, size in files:
+        ancestor = root
+        for part in name.split("/"):
+            ancestor = ancestor / part
+            if ancestor.is_symlink():
+                raise click.ClickException(f"Destination contains a symlink: {name!r}")
+        target = root.joinpath(*name.split("/"))
+        if Path(str(target) + ".aria2").is_symlink():
+            raise click.ClickException(f"Destination contains a symlinked aria2 state file: {name!r}")
+        if not target.resolve().is_relative_to(base):
+            raise click.ClickException(f"File path escapes destination: {name!r}")
+        if target.exists() and target.is_file() and target.stat().st_size != size and not Path(str(target) + ".aria2").exists():
+            raise click.ClickException(f"Existing file has wrong size: {name!r}; move it aside before retrying")
+        if target.exists() and not target.is_file():
+            raise click.ClickException(f"Destination is not a regular file: {name!r}")
+    control = root / ".chathfd"
+    if not control.resolve().is_relative_to(base) or control.is_symlink() or (control / "urls.txt").is_symlink() or (control / "selection.json").is_symlink():
+        raise click.ClickException("Unsafe downloader control path")
+
+
+def _check_selection(root: Path, files: list[tuple[str, int]], identity: dict) -> None:
+    control = root / ".chathfd"
+    receipt = control / "selection.json"
+    if receipt.is_file():
+        try:
+            previous = json.loads(receipt.read_text(encoding="utf-8"))
+        except (OSError, ValueError, UnicodeDecodeError) as exc:
+            raise click.ClickException("Unreadable download selection; choose a fresh destination") from exc
+        if previous != identity:
+            raise click.ClickException("Destination belongs to another repository, endpoint, commit or file selection; choose a fresh destination")
+    elif control.exists() or any(
+        target.exists() or Path(str(target) + ".aria2").exists()
+        for name, _ in files
+        for target in (root.joinpath(*name.split("/")),)
+    ):
+        raise click.ClickException("Existing files have no matching download selection; choose a fresh destination")
+
+
+def download_repo(
+    repo_id: str, repo_type: str, revision: str, endpoint: str, local_dir: Path | None,
+    includes: tuple[str, ...], excludes: tuple[str, ...], connections: int, jobs: int,
+    dry_run: bool, allow_large: bool,
+) -> None:
+    repo_parts = _parts(repo_id, _ID_PART, "repository ID")
+    _parts(revision, _REV_PART, "revision")
+    endpoint = _endpoint(endpoint)
+    root = local_dir or Path.cwd().joinpath(repo_type + "s", *repo_parts)
+    metadata = fetch_metadata(endpoint, repo_id, repo_type, revision)
+    files = _selected_files(metadata, includes, excludes)
+    commit = metadata.get("sha")
+    if not isinstance(commit, str) or not re.fullmatch(r"[0-9a-f]{40}", commit):
+        raise click.ClickException("Repository metadata has no valid commit SHA; refusing unpinned download")
+    _check_paths(root, files)
+    total = sum(size for _, size in files)
+    click.echo(f"{repo_type} {repo_id}@{revision} ({commit}): {len(files)} files, {total} bytes -> {root} (endpoint {endpoint})")
+    for name, size in files:
+        click.echo(f"  {name}: {size} bytes")
+    if dry_run:
+        if total > DEFAULT_MAX_BYTES and not allow_large:
+            click.echo("Selection exceeds 10 GiB; a real download requires --allow-large or narrower --include")
+        click.echo("Dry run: no files downloaded")
+        return
+    if total > DEFAULT_MAX_BYTES and not allow_large:
+        raise click.ClickException("Selected files exceed 10 GiB; narrow --include or pass --allow-large explicitly")
+    identity = {
+        "version": 1, "endpoint": endpoint, "repo_id": repo_id,
+        "repo_type": repo_type, "commit": commit,
+        "files": sorted([name, size] for name, size in files),
+    }
+    _check_selection(root, files, identity)
+    if not shutil.which("aria2c"):
+        raise click.ClickException("aria2c is required; run chathfd setup for instructions or chathfd setup --install")
+    control = root / ".chathfd"
+    control.mkdir(parents=True, exist_ok=True)
+    (control / "selection.json").write_text(json.dumps(identity, sort_keys=True) + "\n", encoding="utf-8")
+    urls = control / "urls.txt"
+    prefix = ["datasets"] if repo_type == "dataset" else []
+    url_prefix = endpoint + "/" + "/".join(quote(part, safe="") for part in [*prefix, *repo_parts, "resolve", commit])
+    pending = []
+    for name, size in files:
+        target = root.joinpath(*name.split("/"))
+        if target.is_file() and target.stat().st_size == size and not Path(str(target) + ".aria2").exists():
+            continue
+        target.parent.mkdir(parents=True, exist_ok=True)
+        url = url_prefix + "/" + "/".join(quote(part, safe="") for part in name.split("/"))
+        pending.extend([url, f" dir={target.parent.resolve()}", f" out={target.name}"])
+    urls.write_text("\n".join(pending) + ("\n" if pending else ""), encoding="utf-8")
+    if pending:
+        command = [
+            "aria2c", "--no-conf=true", "--no-netrc=true", f"--input-file={urls}",
+            "--continue=true", "--auto-file-renaming=false",
+            f"--max-connection-per-server={connections}", f"--split={connections}",
+            f"--max-concurrent-downloads={jobs}", "--file-allocation=none",
+        ]
+        try:
+            subprocess.run(command, check=True)
+        except (OSError, subprocess.CalledProcessError) as exc:
+            raise click.ClickException(f"aria2c failed ({type(exc).__name__}); re-run to resume") from exc
+    for name, size in files:
+        target = root.joinpath(*name.split("/"))
+        if not target.is_file() or target.stat().st_size != size or Path(str(target) + ".aria2").exists():
+            raise click.ClickException(f"Download incomplete or size mismatch: {name!r}")
+    click.echo(f"Verified {len(files)} files by size (not cryptographic hashes)")

chathfd-0.1.3/src/chathfd/setup.py

@@ -0,0 +1,50 @@
+"""Guided installation for the external aria2 download engine."""
+
+from __future__ import annotations
+
+import os
+import shutil
+import subprocess
+
+import click
+from chatstyle import ask_confirm
+
+
+def _install_command() -> list[str] | None:
+    if shutil.which("brew"):
+        return ["brew", "install", "aria2"]
+    for manager, args in (
+        ("apt-get", ["install", "-y", "aria2"]),
+        ("dnf", ["install", "-y", "aria2"]),
+        ("pacman", ["-S", "--noconfirm", "aria2"]),
+    ):
+        if shutil.which(manager):
+            command = [manager, *args]
+            return command if os.geteuid() == 0 else ["sudo", "-n", *command]
+    return None
+
+
+def setup_aria2(install: bool) -> None:
+    path = shutil.which("aria2c")
+    if path:
+        click.echo(f"aria2c ready: {path}")
+        return
+    click.echo("aria2c is not installed; ChatHFD requires it for concurrent downloads.")
+    command = _install_command()
+    if not command:
+        raise click.ClickException("No supported package manager found; install aria2 via your OS package manager")
+    manual = " ".join(part for part in command if part != "-n")
+    if not install:
+        click.echo(f"Suggested command: {manual}")
+        click.echo("Run chathfd setup --install to attempt installation with confirmation.")
+        return
+    if not ask_confirm(f"Run {manual}? This changes system packages", default=False):
+        click.echo("Installation cancelled")
+        return
+    try:
+        subprocess.run(command, check=True)
+    except (OSError, subprocess.CalledProcessError) as exc:
+        raise click.ClickException(f"Installation failed ({type(exc).__name__}); run {manual} interactively if authorization is needed") from exc
+    if not shutil.which("aria2c"):
+        raise click.ClickException("Installer completed but aria2c is not on PATH; check your shell environment")
+    click.echo(f"aria2c ready: {shutil.which('aria2c')}")

{chathfd-0.1.2 → chathfd-0.1.3}/tests/test_cli.py

@@ -5,13 +5,15 @@ from chathfd import __version__
 from chathfd.cli import main
 
 
-EXPECTED_ROOT_TREE = (
+TREE_ROOT = (
     "chathfd\n"
     "├── --help  # Show this message and exit.\n"
     "├── --version  # Show the version and exit.\n"
     "├── --tree  # Print the registered CLI tree and exit.\n"
-    "└── --tree-brief  # Print the registered CLI tree without parameter signatures and exit.\n"
+    "├── --tree-brief  # Print the registered CLI tree without parameter signatures and exit.\n"
 )
+DETAILED_TREE = TREE_ROOT + "├── download <REPO-ID> [--repo-type REPO-TYPE] [--revision REVISION] [--endpoint ENDPOINT] [--local-dir LOCAL-DIR] [--include INCLUDE] [--exclude EXCLUDE] [--connections CONNECTIONS] [--jobs JOBS] [--dry-run] [--allow-large]  # Download a public model or dataset with aria2c.\n└── setup [--install]  # Check the aria2c download engine and offer installation.\n"
+BRIEF_TREE = TREE_ROOT + "├── download  # Download a public model or dataset with aria2c.\n└── setup  # Check the aria2c download engine and offer installation.\n"
 
 
 def test_help_mentions_tree_options():
@@ -29,37 +31,37 @@ def test_version_option_reports_package_version():
     assert f"chathfd, version {__version__}" in result.output
 
 
-def test_tree_reports_registered_root_only_surface():
+def test_tree_reports_registered_download_surface():
     result = CliRunner().invoke(main, ["--tree"])
 
     assert result.exit_code == 0
-    assert result.output == EXPECTED_ROOT_TREE
+    assert result.output == DETAILED_TREE
 
 
-def test_tree_brief_reports_registered_root_only_surface():
+def test_tree_brief_reports_registered_download_surface():
     result = CliRunner().invoke(main, ["--tree-brief"])
 
     assert result.exit_code == 0
-    assert result.output == EXPECTED_ROOT_TREE
+    assert result.output == BRIEF_TREE
 
 
 def test_tree_modes_include_and_omit_command_parameter_signatures():
-    @click.command("download")
+    @click.command("example")
     @click.argument("repo_id")
     @click.option("--revision", help="Repository revision.")
-    def download(repo_id: str, revision: str | None) -> None:
+    def example(repo_id: str, revision: str | None) -> None:
         """Download a repository."""
 
-    main.add_command(download)
+    main.add_command(example)
     try:
         detailed = CliRunner().invoke(main, ["--tree"])
         brief = CliRunner().invoke(main, ["--tree-brief"])
     finally:
-        main.commands.pop("download")
+        main.commands.pop("example")
 
     assert detailed.exit_code == 0
-    assert "download <REPO-ID> [--revision REVISION]  # Download a repository." in detailed.output
+    assert "example <REPO-ID> [--revision REVISION]  # Download a repository." in detailed.output
     assert brief.exit_code == 0
-    assert "download  # Download a repository." in brief.output
+    assert "example  # Download a repository." in brief.output
     assert "<REPO-ID>" not in brief.output
     assert "[--revision REVISION]" not in brief.output

{chathfd-0.1.2 → chathfd-0.1.3}/tests/test_version.py

@@ -2,4 +2,4 @@ from chathfd import __version__
 
 
 def test_version_present():
-    assert __version__ == "0.1.2"
+    assert __version__ == "0.1.3"

chathfd-0.1.2/PKG-INFO

@@ -1,88 +0,0 @@
-Metadata-Version: 2.4
-Name: ChatHFD
-Version: 0.1.2
-Summary: ChatHFD: ChatArch Hugging Face Download tooling package
-Author-email: ChatArch <1073853456@qq.com>
-License-Expression: MIT
-Project-URL: Homepage, https://github.com/ChatArch/ChatHFD
-Project-URL: Repository, https://github.com/ChatArch/ChatHFD
-Project-URL: Documentation, https://arch.gh.wzhecnu.cn/ChatHFD/
-Keywords: chathfd,chatarch,cli
-Classifier: Programming Language :: Python :: 3
-Classifier: Operating System :: OS Independent
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: click>=8.0
-Requires-Dist: chatstyle<0.3.0,>=0.2.0
-Requires-Dist: chatenv<0.3.0,>=0.2.10
-Provides-Extra: dev
-Requires-Dist: build; extra == "dev"
-Requires-Dist: pytest; extra == "dev"
-Requires-Dist: twine; extra == "dev"
-Provides-Extra: docs
-Requires-Dist: mkdocs<2.0,>=1.6; extra == "docs"
-Requires-Dist: mkdocs-material<10.0,>=9.5; extra == "docs"
-Requires-Dist: mkdocs-static-i18n<2.0,>=1.2; extra == "docs"
-Requires-Dist: mike<3.0,>=2.0; extra == "docs"
-Dynamic: license-file
-
-<div align="center">
-    <a href="https://pypi.python.org/pypi/ChatHFD">
-        <img src="https://img.shields.io/pypi/v/ChatHFD.svg" alt="PyPI version" />
-    </a>
-    <a href="https://github.com/ChatArch/ChatHFD/actions/workflows/ci.yml">
-        <img src="https://github.com/ChatArch/ChatHFD/actions/workflows/ci.yml/badge.svg" alt="Tests" />
-    </a>
-    <a href="https://arch.gh.wzhecnu.cn/ChatHFD/">
-        <img src="https://img.shields.io/badge/docs-mkdocs-blue.svg" alt="Documentation" />
-    </a>
-</div>
-
-<div align="center">
-
-[英文版](README.en.md) | [简体中文](README.md)
-</div>
-
-# ChatHFD
-
-ChatHFD 是 ChatArch 的 Hugging Face Download（HFD）工具包入口。当前包保持最小 root-only CLI，用于保留可安装、可发现、可发布的工具壳；实际下载编排能力尚未暴露为子命令。
-
-## 快速开始
-
-```bash
-pip install ChatHFD
-chathfd --help
-chathfd --version
-chathfd --tree
-chathfd --tree-brief
-```
-
-## 当前 CLI 树（`--tree`）
-
-```text
-chathfd
-├── --help  # Show this message and exit.
-├── --version  # Show the version and exit.
-├── --tree  # Print the registered CLI tree and exit.
-└── --tree-brief  # Print the registered CLI tree without parameter signatures and exit.
-```
-
-## CLI 边界
-
-- 当前 CLI 只有根选项，没有业务子命令。
-- `--tree` 和 `--tree-brief` 由 ChatStyle 共享 Click tree runtime 从实际注册面生成，公开根名固定为 `chathfd`。
-- `--tree` 默认保留命令参数签名；`--tree-brief` 省略参数签名，但保留命令节点和描述。当前没有参数化子命令，因此两种模式显示同一组 root-only 节点。
-- 后续新增真实 HFD 下载、镜像、缓存或校验命令时，必须先更新 Click 注册面，再用真实 `chathfd --tree` 和 `chathfd --tree-brief` 同步文档。
-
-## 目录结构
-
-- `src/`：包源码
-- `tests/code-tests/`：代码测试和历史测试迁移
-- `tests/cli-tests/`：真实 CLI 测试，doc-first
-- `tests/mock-cli-tests/`：mock/fake CLI 测试，doc-first
-- `docs/`：长期维护文档，由 MkDocs 构建
-
-## 开发说明
-
-扩展脚手架前，先阅读 `DEVELOP.md` 和 `AGENTS.md`。

chathfd-0.1.2/README.md

@@ -1,59 +0,0 @@
-<div align="center">
-    <a href="https://pypi.python.org/pypi/ChatHFD">
-        <img src="https://img.shields.io/pypi/v/ChatHFD.svg" alt="PyPI version" />
-    </a>
-    <a href="https://github.com/ChatArch/ChatHFD/actions/workflows/ci.yml">
-        <img src="https://github.com/ChatArch/ChatHFD/actions/workflows/ci.yml/badge.svg" alt="Tests" />
-    </a>
-    <a href="https://arch.gh.wzhecnu.cn/ChatHFD/">
-        <img src="https://img.shields.io/badge/docs-mkdocs-blue.svg" alt="Documentation" />
-    </a>
-</div>
-
-<div align="center">
-
-[英文版](README.en.md) | [简体中文](README.md)
-</div>
-
-# ChatHFD
-
-ChatHFD 是 ChatArch 的 Hugging Face Download（HFD）工具包入口。当前包保持最小 root-only CLI，用于保留可安装、可发现、可发布的工具壳；实际下载编排能力尚未暴露为子命令。
-
-## 快速开始
-
-```bash
-pip install ChatHFD
-chathfd --help
-chathfd --version
-chathfd --tree
-chathfd --tree-brief
-```
-
-## 当前 CLI 树（`--tree`）
-
-```text
-chathfd
-├── --help  # Show this message and exit.
-├── --version  # Show the version and exit.
-├── --tree  # Print the registered CLI tree and exit.
-└── --tree-brief  # Print the registered CLI tree without parameter signatures and exit.
-```
-
-## CLI 边界
-
-- 当前 CLI 只有根选项，没有业务子命令。
-- `--tree` 和 `--tree-brief` 由 ChatStyle 共享 Click tree runtime 从实际注册面生成，公开根名固定为 `chathfd`。
-- `--tree` 默认保留命令参数签名；`--tree-brief` 省略参数签名，但保留命令节点和描述。当前没有参数化子命令，因此两种模式显示同一组 root-only 节点。
-- 后续新增真实 HFD 下载、镜像、缓存或校验命令时，必须先更新 Click 注册面，再用真实 `chathfd --tree` 和 `chathfd --tree-brief` 同步文档。
-
-## 目录结构
-
-- `src/`：包源码
-- `tests/code-tests/`：代码测试和历史测试迁移
-- `tests/cli-tests/`：真实 CLI 测试，doc-first
-- `tests/mock-cli-tests/`：mock/fake CLI 测试，doc-first
-- `docs/`：长期维护文档，由 MkDocs 构建
-
-## 开发说明
-
-扩展脚手架前，先阅读 `DEVELOP.md` 和 `AGENTS.md`。

chathfd-0.1.2/src/ChatHFD.egg-info/PKG-INFO

@@ -1,88 +0,0 @@
-Metadata-Version: 2.4
-Name: ChatHFD
-Version: 0.1.2
-Summary: ChatHFD: ChatArch Hugging Face Download tooling package
-Author-email: ChatArch <1073853456@qq.com>
-License-Expression: MIT
-Project-URL: Homepage, https://github.com/ChatArch/ChatHFD
-Project-URL: Repository, https://github.com/ChatArch/ChatHFD
-Project-URL: Documentation, https://arch.gh.wzhecnu.cn/ChatHFD/
-Keywords: chathfd,chatarch,cli
-Classifier: Programming Language :: Python :: 3
-Classifier: Operating System :: OS Independent
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: click>=8.0
-Requires-Dist: chatstyle<0.3.0,>=0.2.0
-Requires-Dist: chatenv<0.3.0,>=0.2.10
-Provides-Extra: dev
-Requires-Dist: build; extra == "dev"
-Requires-Dist: pytest; extra == "dev"
-Requires-Dist: twine; extra == "dev"
-Provides-Extra: docs
-Requires-Dist: mkdocs<2.0,>=1.6; extra == "docs"
-Requires-Dist: mkdocs-material<10.0,>=9.5; extra == "docs"
-Requires-Dist: mkdocs-static-i18n<2.0,>=1.2; extra == "docs"
-Requires-Dist: mike<3.0,>=2.0; extra == "docs"
-Dynamic: license-file
-
-<div align="center">
-    <a href="https://pypi.python.org/pypi/ChatHFD">
-        <img src="https://img.shields.io/pypi/v/ChatHFD.svg" alt="PyPI version" />
-    </a>
-    <a href="https://github.com/ChatArch/ChatHFD/actions/workflows/ci.yml">
-        <img src="https://github.com/ChatArch/ChatHFD/actions/workflows/ci.yml/badge.svg" alt="Tests" />
-    </a>
-    <a href="https://arch.gh.wzhecnu.cn/ChatHFD/">
-        <img src="https://img.shields.io/badge/docs-mkdocs-blue.svg" alt="Documentation" />
-    </a>
-</div>
-
-<div align="center">
-
-[英文版](README.en.md) | [简体中文](README.md)
-</div>
-
-# ChatHFD
-
-ChatHFD 是 ChatArch 的 Hugging Face Download（HFD）工具包入口。当前包保持最小 root-only CLI，用于保留可安装、可发现、可发布的工具壳；实际下载编排能力尚未暴露为子命令。
-
-## 快速开始
-
-```bash
-pip install ChatHFD
-chathfd --help
-chathfd --version
-chathfd --tree
-chathfd --tree-brief
-```
-
-## 当前 CLI 树（`--tree`）
-
-```text
-chathfd
-├── --help  # Show this message and exit.
-├── --version  # Show the version and exit.
-├── --tree  # Print the registered CLI tree and exit.
-└── --tree-brief  # Print the registered CLI tree without parameter signatures and exit.
-```
-
-## CLI 边界
-
-- 当前 CLI 只有根选项，没有业务子命令。
-- `--tree` 和 `--tree-brief` 由 ChatStyle 共享 Click tree runtime 从实际注册面生成，公开根名固定为 `chathfd`。
-- `--tree` 默认保留命令参数签名；`--tree-brief` 省略参数签名，但保留命令节点和描述。当前没有参数化子命令，因此两种模式显示同一组 root-only 节点。
-- 后续新增真实 HFD 下载、镜像、缓存或校验命令时，必须先更新 Click 注册面，再用真实 `chathfd --tree` 和 `chathfd --tree-brief` 同步文档。
-
-## 目录结构
-
-- `src/`：包源码
-- `tests/code-tests/`：代码测试和历史测试迁移
-- `tests/cli-tests/`：真实 CLI 测试，doc-first
-- `tests/mock-cli-tests/`：mock/fake CLI 测试，doc-first
-- `docs/`：长期维护文档，由 MkDocs 构建
-
-## 开发说明
-
-扩展脚手架前，先阅读 `DEVELOP.md` 和 `AGENTS.md`。

chathfd-0.1.2/src/chathfd/cli.py

@@ -1,23 +0,0 @@
-"""CLI entrypoint for chathfd."""
-
-from __future__ import annotations
-
-import click
-from chatstyle import add_tree_option
-
-from chathfd import __version__
-
-
-@click.group(invoke_without_command=True)
-@click.version_option(__version__, prog_name="chathfd")
-@add_tree_option(renderer_options={"root_name": "chathfd"})
-@click.pass_context
-def main(ctx: click.Context) -> None:
-    """chathfd command line interface."""
-    if ctx.invoked_subcommand is None:
-        # Root-only package for now: keep normal Click help behavior when no command is supplied.
-        click.echo(ctx.get_help())
-
-
-if __name__ == "__main__":
-    main()