benchloop-client 0.2.20__tar.gz → 0.2.22__tar.gz

{benchloop_client-0.2.20 → benchloop_client-0.2.22}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: benchloop_client
-Version: 0.2.20
+Version: 0.2.22
 Summary: Benchloop CLI — command line tool for benchmark analysis data (metrics, trajectory, reports, artifacts).
 Requires-Python: >=3.10
 Requires-Dist: httpx>=0.25.2

benchloop_client-0.2.22/RELEASE.md

@@ -0,0 +1,143 @@
+# 发布流程（Release to PyPI）
+
+本项目发布为**手动流程**（无 CI 发布流水线）：本地构建 + `twine` 上传到 PyPI。
+包名为 `benchloop_client`，安装后提供 `bl` / `bmcli` 两个命令。
+
+> 本文档基于 0.2.20 版本的实际发布过程整理，含踩坑记录。
+> 发布流程以本文件为准；README.md 末尾的 “Release new package” 片段仅为最小示例。
+
+## 0. 发布前置检查
+
+- [ ] **功能依赖的后端接口已上线生产**。例如 `bl artifacts download --task-id` 依赖
+  `GET /openapi/jobs/{jobId}/artifacts/download`，后端未部署时用户会收到 404。
+  发布顺序永远是：**后端先上线 → CLI 再发版**；
+- [ ] 功能分支已通过 CR 合入 `master`，本地已切到 `master` 并 `git pull`；
+  （也可以直接从 feature 分支发布，但必须保证发布内容与最终合入 master 的内容一致，
+  并在发布所用 commit 上打 tag 以便回溯）
+- [ ] `git status` 干净——发布内容必须精确对应某个 commit。
+
+## 1. Bump 版本号
+
+PyPI 版本号**不可复用**（传错只能 yank 后发新版本），发布前必须递增：
+
+```bash
+# 查询 PyPI 当前最新版本（注意加官方源，内部镜像有同步延迟）
+pip index versions benchloop_client --index-url https://pypi.org/simple
+```
+
+修改 `pyproject.toml`：
+
+```toml
+[project]
+version = "0.2.XX"   # 递增
+```
+
+提交并推送：
+
+```bash
+git add pyproject.toml
+git commit -m "Release 0.2.XX"
+git push origin master
+```
+
+## 2. 测试、构建、校验（本地零风险，可反复执行）
+
+```bash
+# 全量测试
+uv run pytest -q
+
+# 清理旧产物后构建（防止误传旧版本文件）
+rm -rf dist/
+uv build          # 产出 dist/benchloop_client-0.2.XX.tar.gz + .whl
+
+# 元数据校验，应显示 PASSED
+uvx twine check dist/*
+```
+
+冒烟验证（隔离环境安装 wheel，确认入口和新功能参数存在）：
+
+```bash
+uv run --isolated --no-project --with ./dist/benchloop_client-0.2.XX-py3-none-any.whl \
+  bl --help
+uv run --isolated --no-project --with ./dist/benchloop_client-0.2.XX-py3-none-any.whl \
+  python -c "from importlib.metadata import version; print(version('benchloop_client'))"
+```
+
+## 3. 配置 PyPI Token
+
+认证使用 PyPI API Token（需对 `benchloop_client` 项目有上传权限）：
+
+- username 固定为字面量 `__token__`（这是 PyPI 的约定哨兵值）；
+- password 为完整 token（含 `pypi-` 前缀）。
+
+二选一：
+
+```bash
+# 方式 A：环境变量（当次终端会话有效，注意 upload 必须在同一终端执行）
+export TWINE_USERNAME=__token__
+export TWINE_PASSWORD=pypi-xxxx...
+
+# 方式 B：写 ~/.pypirc（一次配置长期有效）
+cat > ~/.pypirc << 'EOF'
+[pypi]
+username = __token__
+password = pypi-你的完整token
+EOF
+chmod 600 ~/.pypirc
+```
+
+## 4. 上传（唯一不可逆步骤）
+
+```bash
+# 确认 dist/ 中只有本次版本的两个文件，然后：
+uvx --python 3.12 twine upload dist/* --verbose
+```
+
+⚠️ **必须带 `--python 3.12`**（或其他 uv 托管的解释器版本）。
+踩坑记录：`uvx` 默认使用系统 Python，如果系统默认 Python（如 pyenv 3.9）编译时未带
+SSL 模块，会报：
+
+```
+ImportError: Can't connect to HTTPS URL because the SSL module is not available.
+```
+
+上传失败不会在 PyPI 留下任何半成品，换解释器重试即可。
+
+成功标志：
+
+```
+View at:
+https://pypi.org/project/benchloop-client/0.2.XX/
+```
+
+（URL 中 `benchloop-client` 是 PyPI 对 `benchloop_client` 的标准化命名，同一个包。）
+
+## 5. 发布后收尾
+
+```bash
+# 验证 PyPI 生效（用版本专属 URL，包级 JSON 可能返回缓存的旧数据）
+curl -s https://pypi.org/pypi/benchloop_client/0.2.XX/json | python3 -c \
+  "import json,sys; d=json.load(sys.stdin); print(d['info']['version'], [f['filename'] for f in d['urls']])"
+
+# 在发布所用 commit 上打 tag 并推送
+git tag v0.2.XX
+git push origin v0.2.XX
+```
+
+端到端验收（真实数据）：
+
+```bash
+pip install --upgrade benchloop_client --index-url https://pypi.org/simple
+bl artifacts download --task-id <真实taskId> -o ./out
+tar tzf ./out/<taskId>/<jobId>.tgz    # 能列出内容即通过
+```
+
+## 常见问题
+
+| 问题 | 处理 |
+| --- | --- |
+| `ImportError: SSL module is not available` | `uvx --python 3.12 twine ...`，见第 4 节 |
+| 上传报 401/403 | token 无效或无该项目上传权限，换 token 重试（失败无副作用） |
+| `pip install` 报 `externally-managed-environment` | 命中 Homebrew Python 的 PEP 668 保护，用虚拟环境安装，或用 venv 内 pip 的**全路径**（shell alias 可能劫持 `pip3`/`python3` 导致 venv 激活了也不生效） |
+| 用户升级后拿不到新版本 | 公司内部镜像（tbmirror）同步 PyPI 有延迟，升级命令需带 `--index-url https://pypi.org/simple` |
+| 传错版本 | 无法覆盖，在 PyPI 上 yank 该版本后递增版本号重新发布 |

{benchloop_client-0.2.20 → benchloop_client-0.2.22}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "benchloop_client"
-version = "0.2.20"
+version = "0.2.22"
 description = "Benchloop CLI — command line tool for benchmark analysis data (metrics, trajectory, reports, artifacts)."
 readme = "README.md"
 requires-python = ">=3.10"

{benchloop_client-0.2.20 → benchloop_client-0.2.22}/src/aliyun/bmcli/cli.py

@@ -11,6 +11,7 @@ from aliyun.bmcli.commands import (
     ap_cmd as ap_cmd_mod,
     artifacts as artifacts_cmd,
     jobs as jobs_cmd,
+    metadata as metadata_cmd,
     metrics as metrics_cmd,
     overview as overview_cmd,
     pull as pull_cmd,
@@ -62,6 +63,10 @@ def main(
     retry: int = typer.Option(
         2, "--retry", help="Retry count on failure (exponential backoff)."
     ),
+    concurrency: int = typer.Option(
+        5, "--concurrency", min=1,
+        help="Concurrent requests/downloads (use 1 for a per-file progress bar).",
+    ),
     verbose: bool = typer.Option(
         False, "-v", "--verbose", help="Debug logs to stderr."
     ),
@@ -87,6 +92,7 @@ def main(
     )
     ctx.obj = Runtime(
         config=cfg,
+        concurrency=concurrency,
         timeout=timeout,
         retry=retry,
         pretty=pretty,
@@ -130,6 +136,7 @@ app.add_typer(overview_cmd.app, name="overview")
 app.add_typer(tasks_cmd.app, name="tasks")
 app.add_typer(jobs_cmd.app, name="jobs")
 app.add_typer(artifacts_cmd.app, name="artifacts")
+app.add_typer(metadata_cmd.app, name="metadata")
 app.command(
     "ap-cmd",
     context_settings={"allow_extra_args": True, "ignore_unknown_options": True},

{benchloop_client-0.2.20 → benchloop_client-0.2.22}/src/aliyun/bmcli/client.py

@@ -30,6 +30,14 @@ class ApiError(ClientError):
         super().__init__(f"API error {status}: {body}")
 
 
+class ArtifactMissingError(ClientError):
+    """Raised when a download endpoint reports the artifact does not exist.
+
+    覆盖两种形态：后端直接返回 404；以及网关把缺产物的 404 重定向成
+    HTML 错误页（HTTP 200 + text/html）。两种情况都不应把响应体写盘。
+    """
+
+
 class ApiClient:
     """Thin wrapper over ``httpx.Client`` for the benchmark openapi."""
 
@@ -135,8 +143,17 @@ class ApiClient:
             raise last_exc
         raise NetworkError("request failed")  # pragma: no cover
 
-    def download_file(self, url: str, dest_path: str | Path) -> Path:
-        """Stream ``url`` to ``dest_path`` on disk. Returns the path written."""
+    def download_file(
+        self,
+        url: str,
+        dest_path: str | Path,
+        progress: Optional[Any] = None,
+    ) -> Path:
+        """Stream ``url`` to ``dest_path`` on disk. Returns the path written.
+
+        ``progress``（可选）需提供 ``begin(total)`` / ``advance(nbytes)`` 两个
+        方法（如 ``output.TransferStream``）；每次重试会重新 ``begin`` 归零。
+        """
         dest = Path(dest_path)
         dest.parent.mkdir(parents=True, exist_ok=True)
         last_exc: Optional[Exception] = None
@@ -146,12 +163,31 @@ class ApiClient:
                     with client.stream("GET", self._url(url), headers=self._headers()) as resp:
                         if resp.status_code == 401:
                             raise AuthError("authentication failed (HTTP 401)")
+                        if resp.status_code == 404:
+                            body = self._safe_stream_body(resp)
+                            raise ArtifactMissingError(
+                                f"artifact not found (HTTP 404): {body[:200]}"
+                            )
                         if resp.status_code >= 400:
                             body = self._safe_stream_body(resp)
                             raise ApiError(resp.status_code, body)
+                        content_type = (resp.headers.get("content-type") or "").lower()
+                        if "text/html" in content_type:
+                            # 网关会把缺产物的 404 重定向成 HTML 错误页
+                            # （HTTP 200），不能把错误页当产物写盘。
+                            raise ArtifactMissingError(
+                                "download returned an html error page "
+                                "(artifact missing on OSS?)"
+                            )
+                        if progress is not None:
+                            raw_length = resp.headers.get("content-length") or ""
+                            total = int(raw_length) if raw_length.isdigit() else None
+                            progress.begin(total)
                         with dest.open("wb") as fh:
                             for chunk in resp.iter_bytes(chunk_size=65536):
                                 fh.write(chunk)
+                                if progress is not None:
+                                    progress.advance(len(chunk))
                 return dest
             except (httpx.TimeoutException, httpx.TransportError) as exc:
                 last_exc = NetworkError(str(exc))

{benchloop_client-0.2.20 → benchloop_client-0.2.22}/src/aliyun/bmcli/commands/_common.py

@@ -2,6 +2,7 @@
 
 from __future__ import annotations
 
+import tarfile
 from pathlib import Path
 from typing import Optional
 
@@ -41,6 +42,18 @@ def resolve_output_dir(runtime_output: str, override: Optional[str]) -> Path:
     return Path(override) if override else Path(runtime_output)
 
 
+def write_empty_tgz(dest: Path) -> Path:
+    """Write a valid empty .tgz placeholder for jobs whose artifact is missing.
+
+    产物在 OSS 上不存在时落一个合法空包，而不是把错误页 HTML 写盘，
+    保证下游批量解包不会被脏文件打断。
+    """
+    dest.parent.mkdir(parents=True, exist_ok=True)
+    with tarfile.open(dest, "w:gz"):
+        pass
+    return dest
+
+
 def resolve_task_id(
     task_id: Optional[str], groups: list[str], jobs: list[str],
 ) -> str:

{benchloop_client-0.2.20 → benchloop_client-0.2.22}/src/aliyun/bmcli/commands/artifacts.py

@@ -21,6 +21,7 @@ from aliyun.bmcli.commands._common import (
     resolve_output_dir,
     resolve_task_id,
     task_job_ids,
+    write_empty_tgz,
 )
 from aliyun.bmcli.commands.group_data import (
     fetch_all_jobs,
@@ -28,6 +29,7 @@ from aliyun.bmcli.commands.group_data import (
     job_ids,
     raise_for_failures,
 )
+from aliyun.bmcli.client import ArtifactMissingError
 from aliyun.bmcli.exit_codes import PARTIAL
 from aliyun.bmcli.runtime import get_runtime, report_and_exit, safe_run_concurrent
 
@@ -161,6 +163,9 @@ def download(
 
         # OSS 公网加签已禁用：不再通过 listing 拿加签 URL 直连 OSS，而是逐 job
         # 请求后端代理接口，由后端从 OSS 内网读取并流式转发 result.tgz。
+        transfer_bar = output.TransferProgressBar("task artifacts", len(task_jobs))
+        transfer_bar.start()
+
         def task_job_worker(job_id: str) -> str:
             try:
                 # Encode the snapshot job_id before using it as a local file
@@ -173,10 +178,20 @@ def download(
                 if skip_existing and dest.exists():
                     output.log(f"skip existing {dest}")
                     return str(dest)
-                rt.client.download_file(
-                    f"/openapi/jobs/{quote(job_id, safe='')}/artifacts/download",
-                    dest,
-                )
+                stream = transfer_bar.stream(dest.name)
+                try:
+                    rt.client.download_file(
+                        f"/openapi/jobs/{quote(job_id, safe='')}/artifacts/download",
+                        dest,
+                        progress=stream,
+                    )
+                except ArtifactMissingError as exc:
+                    # 产物在 OSS 上不存在：落合法空包占位，不计入失败。
+                    write_empty_tgz(dest)
+                    output.warn(f"no artifact for job={job_id}, wrote empty {dest} ({exc})")
+                    return str(dest)
+                finally:
+                    stream.finish()
                 output.log(f"downloaded {dest}")
                 return str(dest)
             finally:
@@ -184,7 +199,10 @@ def download(
                 # the visible task progress forward.
                 progress.advance(job_id)
 
-        result = safe_run_concurrent(task_jobs, task_job_worker, rt.concurrency)
+        try:
+            result = safe_run_concurrent(task_jobs, task_job_worker, rt.concurrency)
+        finally:
+            transfer_bar.close()
         report_and_exit(result, emit=False)
         return
 
@@ -199,6 +217,10 @@ def download(
                 action="processed",
             )
             progress.start()
+            transfer_bar = output.TransferProgressBar(
+                f"group {group_id} artifacts", len(ids)
+            )
+            transfer_bar.start()
 
             def process_job(client, job_id: str):
                 failures = []
@@ -223,18 +245,39 @@ def download(
                         if skip_existing and dest.exists():
                             output.log(f"skip existing {dest}")
                             continue
+                        stream = transfer_bar.stream(f"{job_id}/{dest.name}")
                         try:
-                            client.download_file(artifact["url"], dest)
+                            # 与 --task-id 链路一致：不直连 OSS 加签 URL，改走后端
+                            # 代理接口流式下载；listing 的 url 仅作产物存在信号。
+                            client.download_file(
+                                f"/openapi/jobs/{quote(job_id, safe='')}"
+                                "/artifacts/download",
+                                dest,
+                                progress=stream,
+                            )
                             output.log(f"downloaded {dest}")
+                        except ArtifactMissingError as exc:
+                            write_empty_tgz(dest)
+                            output.warn(
+                                f"no artifact for job={job_id}, "
+                                f"wrote empty {dest} ({exc})"
+                            )
                         except Exception as exc:  # noqa: BLE001 - retain other jobs
                             failures.append(
                                 (f"{job_id}:{artifact.get('name')}", str(exc))
                             )
+                        finally:
+                            stream.finish()
                     return failures
                 finally:
                     progress.advance(job_id)
 
-            result = fetch_job_resources(rt.client, ids, process_job, rt.concurrency)
+            try:
+                result = fetch_job_resources(
+                    rt.client, ids, process_job, rt.concurrency
+                )
+            finally:
+                transfer_bar.close()
             if result.failures and not result.items:
                 raise_for_failures("artifact listings", result.failures)
             nested = [
@@ -336,30 +379,49 @@ def download(
 
     def _download_one(art: dict[str, Any]) -> Optional[tuple[str, str]]:
         name = art.get("name") or "artifact"
-        url = art.get("url")
-        if not url:  # already reported above; defensive guard
+        if not art.get("url"):  # already reported above; defensive guard
             return None
         dest = _dest_for(art)
         if skip_existing and dest.exists():
             output.log(f"skip existing {dest}")
             download_progress.advance(str(dest))
             return None
+        job_id = str(art.get("job_id") or "")
+        stream = transfer_bar.stream(f"{job_id}/{name}" if job_id else name)
         try:
-            rt.client.download_file(url, dest)
+            # 与 --task-id 链路一致：改走后端代理接口下载，listing 返回的
+            # 加签 url 仅作产物存在信号，不再直连 OSS。
+            rt.client.download_file(
+                f"/openapi/jobs/{quote(job_id, safe='')}/artifacts/download",
+                dest,
+                progress=stream,
+            )
             output.log(f"downloaded {dest}")
             download_progress.advance(str(dest))
             return None
+        except ArtifactMissingError as exc:
+            write_empty_tgz(dest)
+            output.warn(f"no artifact for job={art.get('job_id')}, wrote empty {dest} ({exc})")
+            download_progress.advance(str(dest))
+            return None
         except Exception as exc:  # noqa: BLE001 - per-artifact failure tolerated
             return (f"{art.get('job_id')}:{name}", str(exc))
+        finally:
+            stream.finish()
 
     if downloadable:
+        transfer_bar = output.TransferProgressBar(progress_label, len(downloadable))
+        transfer_bar.start()
         max_workers = max(1, min(rt.concurrency, len(downloadable)))
-        with ThreadPoolExecutor(max_workers=max_workers) as pool:
-            futures = [pool.submit(_download_one, art) for art in downloadable]
-            for future in as_completed(futures):
-                result = future.result()
-                if result is not None:
-                    failures.append(result)
+        try:
+            with ThreadPoolExecutor(max_workers=max_workers) as pool:
+                futures = [pool.submit(_download_one, art) for art in downloadable]
+                for future in as_completed(futures):
+                    result = future.result()
+                    if result is not None:
+                        failures.append(result)
+        finally:
+            transfer_bar.close()
 
     listing.failures = failures
     report_and_exit(listing, emit=False)

benchloop_client-0.2.22/src/aliyun/bmcli/commands/metadata.py

@@ -0,0 +1,175 @@
+"""`bmcli metadata` — job metadata for a CLI download task (task-id only)."""
+
+from __future__ import annotations
+
+import json
+import tempfile
+import zipfile
+from pathlib import Path
+from typing import Optional
+from urllib.parse import quote
+
+import typer
+
+from aliyun.bmcli import output
+from aliyun.bmcli.commands._common import (
+    OutputDirOption,
+    SkipExistingOption,
+    resolve_output_dir,
+    resolve_task_id,
+    task_job_ids,
+)
+from aliyun.bmcli.exit_codes import OK, PARTIAL
+from aliyun.bmcli.runtime import get_runtime, report_and_exit, safe_run_concurrent
+
+app = typer.Typer(help="Job metadata (download task).", no_args_is_help=True)
+
+
+def _write_metadata_archive(
+    archive_path: Path,
+    task_payload: dict,
+    task_id: str,
+    downloaded: list[tuple[str, Path]],
+    failures: list[tuple[str, str]],
+    *,
+    pretty: bool,
+) -> Path:
+    """Create the metadata ZIP atomically: metadata/{job_id}.json + manifest.json."""
+    manifest = {
+        "task_id": task_id,
+        "group_id": task_payload.get("group_id"),
+        "created_at": task_payload.get("created_at"),
+        "total_jobs": len(downloaded) + len(failures),
+        "successful_jobs": len(downloaded),
+        "failed_jobs": len(failures),
+        "success_job_ids": [job_id for job_id, _path in downloaded],
+        "failures": [{"job_id": job_id, "error": error} for job_id, error in failures],
+    }
+    indent = 2 if pretty else None
+    separators = None if pretty else (",", ":")
+    manifest_text = (
+        json.dumps(
+            manifest,
+            ensure_ascii=False,
+            indent=indent,
+            separators=separators,
+        )
+        + "\n"
+    )
+
+    # 与轨迹任务包同一套原子写法：临时 zip 落在目标目录内，成功后 replace。
+    with tempfile.NamedTemporaryFile(
+        prefix=f".{archive_path.name}.",
+        suffix=".tmp",
+        dir=archive_path.parent,
+        delete=False,
+    ) as tmp_archive:
+        tmp_archive_path = Path(tmp_archive.name)
+    try:
+        with zipfile.ZipFile(
+            tmp_archive_path,
+            mode="w",
+            compression=zipfile.ZIP_DEFLATED,
+            allowZip64=True,
+        ) as archive:
+            for job_id, path in downloaded:
+                file_job_id = quote(job_id, safe="-_.")
+                archive.write(path, arcname=f"metadata/{file_job_id}.json")
+            archive.writestr("manifest.json", manifest_text)
+        tmp_archive_path.replace(archive_path)
+    except Exception:
+        tmp_archive_path.unlink(missing_ok=True)
+        raise
+    return archive_path
+
+
+@app.command("download")
+def download(
+    ctx: typer.Context,
+    task_id: str = typer.Option(
+        ...,
+        "--task-id",
+        help="Download task ID created by the benchmark analysis page.",
+    ),
+    output_dir: Optional[str] = OutputDirOption,
+    skip_existing: bool = SkipExistingOption,
+) -> None:
+    """Download per-job metadata for a task into {task_id}_metadata.zip."""
+    rt = get_runtime(ctx)
+    # 复用统一校验：task_id 不得包含路径分隔符，保证归档留在输出目录内。
+    task_id = resolve_task_id(task_id, [], [])
+    root = resolve_output_dir(rt.output_dir, output_dir)
+
+    archive_path = root / f"{task_id}_metadata.zip"
+    if skip_existing and archive_path.exists():
+        output.log(f"skip existing {archive_path}")
+        raise typer.Exit(code=OK)
+
+    task_result = safe_run_concurrent(
+        [task_id],
+        lambda tid: rt.client.get_json(
+            f"/openapi/trajectory-download-tasks/{quote(tid, safe='')}"
+        ),
+        1,
+    )
+    if task_result.failures:
+        report_and_exit(task_result, emit=False)
+    if not task_result.items:
+        output.error("download task response is empty")
+        raise typer.Exit(code=PARTIAL)
+
+    task_payload = task_result.items[0]
+    try:
+        task_jobs = task_job_ids(task_payload)
+    except ValueError as exc:
+        output.error(str(exc))
+        raise typer.Exit(code=PARTIAL) from exc
+
+    output.ensure_dir(root)
+    progress = output.ProgressCounter(
+        "task metadata downloads",
+        len(task_jobs),
+        "jobs",
+        action="processed",
+        show_eta=True,
+    )
+    progress.start()
+
+    with tempfile.TemporaryDirectory(
+        prefix=f".{task_id}-metadata-", dir=root
+    ) as temp_dir:
+        temp_root = Path(temp_dir)
+
+        def task_job_worker(job_id: str) -> tuple[str, Path]:
+            try:
+                meta = rt.client.get_json(
+                    f"/openapi/jobs/{quote(job_id, safe='')}/meta"
+                )
+                # job_id comes from the server-side task snapshot. Encode it
+                # before using it as a local filename so an unexpected slash
+                # or traversal segment can never escape the temporary folder.
+                file_job_id = quote(job_id, safe="-_.")
+                dest = temp_root / f"{file_job_id}.json"
+                output.write_json_file(dest, meta, pretty=rt.pretty)
+                return job_id, dest
+            finally:
+                # A failed request is still a completed attempt and must move
+                # the visible task progress forward.
+                progress.advance(job_id)
+
+        result = safe_run_concurrent(task_jobs, task_job_worker, rt.concurrency)
+        if not result.items:
+            report_and_exit(result, emit=False)
+
+        output.progress(f"creating archive {archive_path}")
+        _write_metadata_archive(
+            archive_path,
+            task_payload,
+            task_id,
+            result.items,
+            result.failures,
+            pretty=rt.pretty,
+        )
+
+    output.log(f"wrote {archive_path}")
+    report_and_exit(result, emit=False)