dexverse-cli 0.0.4a1__tar.gz → 0.0.4.post2__tar.gz

{dexverse_cli-0.0.4a1 → dexverse_cli-0.0.4.post2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: dexverse-cli
-Version: 0.0.4a1
+Version: 0.0.4.post2
 Summary: DexVerse dataset downloader for S3-compatible robot training datasets
 Author: DexVerse
 License: Apache-2.0
@@ -104,6 +104,9 @@ D:\DexVerse\20260429_180359.hdf5
 items resolve to the same local path (for example, repeated `file_name`
 values), the first item wins and later duplicates are ignored.
 
+For compatibility, the CLI also accepts the legacy `task_instruction` field;
+both names must not appear in the same file entry.
+
 Include `file_size` (and preferably `sha256`) for large files. The HTTP backend
 also learns the total size from `Content-Length`/`Content-Range`, but manifest
 metadata lets the CLI verify an existing local file before making a request.
@@ -134,7 +137,7 @@ dexverse-dl repair  --manifest PATH --out /data/dexverse [--backend http|python|
 
 下载完成后，把落盘的 `.hdf5` 还原为原始录制结构（逐帧 JPEG + `pose_record_*.json`
 + `metadata.jsonl`）。如果对应 manifest 带有 `additional_info`，还会自动生成
-`additional_info.json`：
+`task_instruction.json`（保持算法侧兼容）：
 
 ```bash
 # 单文件
@@ -153,12 +156,13 @@ dexverse-dl convert --input-dir /data/dexverse -o /data/dexverse/restored
   hand/right/000000.jpg ...
   pose_record_20260429_180359.json
   metadata.jsonl
-  additional_info.json
+  task_instruction.json
 ```
 
-`additional_info.json` 的内容就是 manifest 中的对象本身，不增加外层包装。
-下载器会把转换所需的映射保存在 `.dexverse/additional-info/`，无需为
-`convert` 增加额外参数。独立来源且没有该隐藏元数据的 HDF5 仍按原行为还原。
+`task_instruction.json` 的内容就是 manifest 中的对象本身，不增加外层包装。
+下载器会把转换所需的映射保存在 `.dexverse/task-instructions/`，无需为
+`convert` 增加额外参数。设备侧 `data_protocol` 不会写入算法侧
+`task_instruction.json`；独立来源且没有该隐藏元数据的 HDF5 仍按原行为还原。
 
 该命令依赖 `h5py` / `numpy` / `av` / `pillow`，已列入核心依赖，安装时随包装上。
 
@@ -180,7 +184,7 @@ State files live under `<out>/.dexverse/` and never mix into the dataset tree:
 
 ```text
 /data/dexverse/
-  .dexverse/additional-info/
+  .dexverse/task-instructions/
     20260429_180359.hdf5.json
   .dexverse/tasks/<taskId>/
     manifest.json

{dexverse_cli-0.0.4a1 → dexverse_cli-0.0.4.post2}/README.md

@@ -74,6 +74,9 @@ D:\DexVerse\20260429_180359.hdf5
 items resolve to the same local path (for example, repeated `file_name`
 values), the first item wins and later duplicates are ignored.
 
+For compatibility, the CLI also accepts the legacy `task_instruction` field;
+both names must not appear in the same file entry.
+
 Include `file_size` (and preferably `sha256`) for large files. The HTTP backend
 also learns the total size from `Content-Length`/`Content-Range`, but manifest
 metadata lets the CLI verify an existing local file before making a request.
@@ -104,7 +107,7 @@ dexverse-dl repair  --manifest PATH --out /data/dexverse [--backend http|python|
 
 下载完成后，把落盘的 `.hdf5` 还原为原始录制结构（逐帧 JPEG + `pose_record_*.json`
 + `metadata.jsonl`）。如果对应 manifest 带有 `additional_info`，还会自动生成
-`additional_info.json`：
+`task_instruction.json`（保持算法侧兼容）：
 
 ```bash
 # 单文件
@@ -123,12 +126,13 @@ dexverse-dl convert --input-dir /data/dexverse -o /data/dexverse/restored
   hand/right/000000.jpg ...
   pose_record_20260429_180359.json
   metadata.jsonl
-  additional_info.json
+  task_instruction.json
 ```
 
-`additional_info.json` 的内容就是 manifest 中的对象本身，不增加外层包装。
-下载器会把转换所需的映射保存在 `.dexverse/additional-info/`，无需为
-`convert` 增加额外参数。独立来源且没有该隐藏元数据的 HDF5 仍按原行为还原。
+`task_instruction.json` 的内容就是 manifest 中的对象本身，不增加外层包装。
+下载器会把转换所需的映射保存在 `.dexverse/task-instructions/`，无需为
+`convert` 增加额外参数。设备侧 `data_protocol` 不会写入算法侧
+`task_instruction.json`；独立来源且没有该隐藏元数据的 HDF5 仍按原行为还原。
 
 该命令依赖 `h5py` / `numpy` / `av` / `pillow`，已列入核心依赖，安装时随包装上。
 
@@ -150,7 +154,7 @@ State files live under `<out>/.dexverse/` and never mix into the dataset tree:
 
 ```text
 /data/dexverse/
-  .dexverse/additional-info/
+  .dexverse/task-instructions/
     20260429_180359.hdf5.json
   .dexverse/tasks/<taskId>/
     manifest.json

dexverse_cli-0.0.4.post2/decision-traces/2026-08-12_1613_pypi_0_0_4_release_trace.md

@@ -0,0 +1,84 @@
+# 2026-08-12 PyPI 0.0.4 Release Boundary
+
+## Background
+
+`dexverse-cli` was prepared for a PyPI 0.0.4 release. The release must remain
+the dataset downloader and must not ship the co-located device cloud-upload
+module or the checked-in HDF5 fixture.
+
+## Problem
+
+The original Hatch package selection included all files under `dexverse`, so
+both wheel and sdist included `dexverse/dexe_cloud_upload`. The source
+distribution also included `648584.hdf5` (about 41.6 MiB). In addition, the
+public `dexverse.__version__` lookup used the import package name instead of
+the distribution name, so it returned `0.0.0+unknown` after installation.
+
+## Evidence And Source Analysis
+
+- The pre-release build contained 25 `dexe_cloud_upload` entries in both wheel
+  and sdist, and the sdist contained `648584.hdf5`.
+- The 0.0.4 build excluded `/dexverse/dexe_cloud_upload/**` and
+  `/648584.hdf5`; the resulting wheel had 23 entries and the sdist had 53,
+  with zero entries for either exclusion target.
+- `pyproject.toml` names the distribution `dexverse-cli`; changing
+  `version("dexverse")` to `version("dexverse-cli")` made the installed
+  public version equal the distribution version.
+- The local suite passed 110 tests. Twine metadata checks passed. Clean Python
+  3.10 wheel and Python 3.11 sdist installs succeeded, as did a final PyPI
+  Simple Index installation of `dexverse-cli==0.0.4`.
+- A real HDF5 download and deep SHA-256 verification succeeded with the PyPI
+  alpha. `repair --deep` does not re-download a same-size SHA-mismatched file,
+  but the same behavior and identical `repair()` implementation were confirmed
+  in PyPI 0.0.3; it was accepted as an existing issue, not a 0.0.4 regression.
+
+## Clarified Constraints
+
+- The 0.0.4 release must not include `dexverse/dexe_cloud_upload`.
+- The 0.0.4 release must not include `648584.hdf5`.
+- P1 documentation, license, and Docker concerns were intentionally deferred
+  for the later repository refactor.
+- PyPI files are immutable per version, so alpha validation preceded the
+  formal release.
+- The trace is local-only and must not contain package-upload credentials.
+
+## Final Decision
+
+PyPI `dexverse-cli` 0.0.4 was published with the exclusion rules and corrected
+distribution-name version lookup. The release contains the downloader CLI/SDK,
+but not the cloud-upload module or HDF5 fixture. The pre-existing deep-repair
+limitation is not treated as a release blocker for 0.0.4.
+
+Future releases must inspect both wheel and sdist contents before upload and
+must keep `dexverse.__version__` aligned with the `dexverse-cli` distribution
+metadata. Do not reintroduce the excluded paths into downloader distributions
+unless product scope explicitly changes.
+
+## Current Status
+
+- PyPI 0.0.4 wheel and sdist published successfully on 2026-08-12.
+- PyPI installation, version check, exclusion-content check, and CLI help
+  smoke test passed.
+- The local release edits are deliberately being reverted after publication at
+  user request. Consequently, the checkout will not reproduce the published
+  0.0.4 artifact until the release changes are committed or reapplied.
+- This trace is intentionally local and uncommitted.
+
+## Revisit Triggers
+
+- Start of the planned repository refactor or any new package release.
+- A decision to ship the cloud-upload module as a separate distribution or to
+  include it in this package.
+- A decision to fix `repair --deep` behavior, which requires a regression test
+  for same-size SHA-256 corruption and forced re-download.
+- A request to align the Git checkout, tag, or release notes with the published
+  PyPI 0.0.4 artifact.
+
+## Privacy Class
+
+trace_privacy_class: local_raw_trace
+
+## TraceGym Data Capture
+
+TraceGym metadata omitted: this is a repository-specific release provenance
+record, not an evaluated reusable workflow or benchmark artifact.

{dexverse_cli-0.0.4a1 → dexverse_cli-0.0.4.post2}/dexverse/convert/hdf5_to_raw.py

@@ -31,11 +31,12 @@ import h5py
 import numpy as np
 
 from ..downloader import state as download_state
+from ..manifest import algorithm_task_instruction
 # 复用同包子模块的解码能力
 from .image_video_encode import decode_mp4_bytes_to_rgb_images
 
 
-ADDITIONAL_INFO_FILENAME = "additional_info.json"
+TASK_INSTRUCTION_FILENAME = "task_instruction.json"
 
 
 def _find_download_root(hdf5_path: Path) -> Optional[Path]:
@@ -47,26 +48,33 @@ def _find_download_root(hdf5_path: Path) -> Optional[Path]:
     return None
 
 
-def _sync_restored_additional_info(
+def _sync_restored_task_instruction(
     hdf5_path: Path,
     output_dir: Path,
 ) -> None:
-    """Copy a downloaded file's hidden additional info into its restored session."""
+    """Restore metadata to the legacy algorithm-facing output name."""
     download_root = _find_download_root(hdf5_path)
     if download_root is None:
         return
 
     resolved_hdf5 = Path(hdf5_path).expanduser().resolve()
     relpath = resolved_hdf5.relative_to(download_root).as_posix()
-    source = download_state.additional_info_path(download_root, relpath)
-    destination = Path(output_dir) / ADDITIONAL_INFO_FILENAME
+    source = download_state.task_instruction_path(download_root, relpath)
+    if not source.is_file():
+        source = download_state.additional_info_path(download_root, relpath)
+    destination = Path(output_dir) / TASK_INSTRUCTION_FILENAME
 
     if source.is_file():
-        additional_info = json.loads(source.read_text(encoding="utf-8"))
-        if not isinstance(additional_info, dict):
-            raise ValueError(f"additional info must be a JSON object: {source}")
+        task_instruction = json.loads(source.read_text(encoding="utf-8"))
+        if not isinstance(task_instruction, dict):
+            raise ValueError(f"task instruction must be a JSON object: {source}")
         destination.write_text(
-            json.dumps(additional_info, ensure_ascii=False, indent=2) + "\n",
+            json.dumps(
+                algorithm_task_instruction(task_instruction),
+                ensure_ascii=False,
+                indent=2,
+            )
+            + "\n",
             encoding="utf-8",
         )
     else:
@@ -249,7 +257,7 @@ class HDF5ToRawConverter:
                         img = Image.open(_io.BytesIO(jpg_bytes)).convert("RGB")
                         img.save(cam_dir / f"{i:06d}.jpg", "JPEG", quality=95)
 
-        _sync_restored_additional_info(hdf5_path, output_dir)
+        _sync_restored_task_instruction(hdf5_path, output_dir)
         return frames_count
 
     def _extract_frames(self, h5_file: h5py.File) -> List[AlignedFrame]:

{dexverse_cli-0.0.4a1 → dexverse_cli-0.0.4.post2}/dexverse/downloader/manager.py

@@ -20,7 +20,7 @@ from dataclasses import dataclass, field
 from pathlib import Path
 from typing import Dict, List, Optional, Union
 
-from ..manifest import FileSpec, Manifest, resolve_dest
+from ..manifest import FileSpec, Manifest, algorithm_task_instruction, resolve_dest
 from . import state as st
 from .base import Backend, ProgressReporter
 from .http_backend import HttpBackend, _HTTP_BACKEND_NAMES
@@ -43,26 +43,26 @@ def _persist_manifest_copy(out: Path, task_id: str, manifest: Manifest) -> None:
         pass
 
 
-def _sync_additional_info(
+def _sync_task_instructions(
     out: Path,
     manifest: Manifest,
     task_state: st.TaskState,
 ) -> None:
-    """Synchronize per-file additional info after download/verification."""
+    """Write per-file metadata to the legacy local compatibility path."""
     successful_statuses = {st.DOWNLOADED, st.SKIPPED, st.VERIFIED}
     for file_spec in manifest.files:
         key = file_spec.effective_key()
         file_state = task_state.files.get(key)
-        info_path = st.additional_info_path(out, file_spec.relpath())
+        instruction_path = st.task_instruction_path(out, file_spec.relpath())
         if (
             file_state is not None
             and file_state.status in successful_statuses
             and file_spec.additionalInfo is not None
         ):
-            info_path.parent.mkdir(parents=True, exist_ok=True)
-            info_path.write_text(
+            instruction_path.parent.mkdir(parents=True, exist_ok=True)
+            instruction_path.write_text(
                 json.dumps(
-                    file_spec.additionalInfo,
+                    algorithm_task_instruction(file_spec.additionalInfo),
                     ensure_ascii=False,
                     indent=2,
                 )
@@ -70,7 +70,7 @@ def _sync_additional_info(
                 encoding="utf-8",
             )
         else:
-            info_path.unlink(missing_ok=True)
+            instruction_path.unlink(missing_ok=True)
 
 
 def get_backend(name: str, **kwargs) -> Backend:
@@ -280,7 +280,7 @@ class DownloadManager:
                 verified += 1
                 task_state.mark(key, st.VERIFIED)
 
-        _sync_additional_info(self.out, manifest, task_state)
+        _sync_task_instructions(self.out, manifest, task_state)
         task_state.finish()
         task_state.save()
 

{dexverse_cli-0.0.4a1 → dexverse_cli-0.0.4.post2}/dexverse/downloader/state.py

@@ -17,6 +17,7 @@ from ..manifest import Manifest, resolve_dest
 
 STATE_DIRNAME = ".dexverse"
 TASKS_DIRNAME = "tasks"
+TASK_INSTRUCTIONS_DIRNAME = "task-instructions"
 ADDITIONAL_INFO_DIRNAME = "additional-info"
 
 # file statuses
@@ -36,13 +37,18 @@ def task_dir(out: Path, task_id: str) -> Path:
     return dexverse_root(out) / TASKS_DIRNAME / task_id
 
 
-def additional_info_dir(out: Path) -> Path:
-    return dexverse_root(out) / ADDITIONAL_INFO_DIRNAME
+def task_instructions_dir(out: Path) -> Path:
+    return dexverse_root(out) / TASK_INSTRUCTIONS_DIRNAME
+
+
+def task_instruction_path(out: Path, relpath: str) -> Path:
+    """Legacy hidden metadata path written for algorithm compatibility."""
+    return resolve_dest(task_instructions_dir(out), f"{relpath}.json")
 
 
 def additional_info_path(out: Path, relpath: str) -> Path:
-    """Hidden metadata path for one downloaded dataset file."""
-    return resolve_dest(additional_info_dir(out), f"{relpath}.json")
+    """Read-only bridge for state written by release 32ffa8e."""
+    return resolve_dest(dexverse_root(out) / ADDITIONAL_INFO_DIRNAME, f"{relpath}.json")
 
 
 def state_file_path(out: Path, task_id: str) -> Path:

{dexverse_cli-0.0.4a1 → dexverse_cli-0.0.4.post2}/dexverse/manifest.py

@@ -22,6 +22,13 @@ from typing import Any, Dict, List, Optional
 from pydantic import AliasChoices, BaseModel, ConfigDict, Field, model_validator
 
 
+def algorithm_task_instruction(additional_info: Dict[str, Any]) -> Dict[str, Any]:
+    """Return algorithm-facing task metadata without device-only protocol data."""
+    task_instruction = dict(additional_info)
+    task_instruction.pop("data_protocol", None)
+    return task_instruction
+
+
 # --------------------------------------------------------------------------- #
 # Path helpers
 # --------------------------------------------------------------------------- #
@@ -162,31 +169,38 @@ class FileSpec(BaseModel):
         None, validation_alias=AliasChoices("presignedUrl", "download_url", "url", "downloadUrl"))
     additionalInfo: Optional[Dict[str, Any]] = Field(
         None,
-        validation_alias=AliasChoices("additionalInfo", "additional_info"),
+        validation_alias=AliasChoices(
+            "additionalInfo",
+            "additional_info",
+            "taskInstruction",
+            "task_instruction",
+        ),
     )
 
     @model_validator(mode="before")
     @classmethod
     def validate_metadata_fields(cls, data: Any) -> Any:
-        """Reject ambiguous aliases and removed names before extras are kept."""
+        """Reject ambiguous metadata aliases before extras are kept."""
         if isinstance(data, dict):
-            if "additional_info" in data and "additionalInfo" in data:
-                raise ValueError(
-                    "fields 'additional_info' and 'additionalInfo' cannot be "
-                    "used together; provide exactly one"
-                )
-            removed = [
-                name
-                for name in ("task_instruction", "taskInstruction")
-                if name in data
-            ]
-            if removed:
-                names = ", ".join(repr(name) for name in removed)
+            metadata_fields = (
+                "additional_info",
+                "additionalInfo",
+                "task_instruction",
+                "taskInstruction",
+            )
+            present = [name for name in metadata_fields if name in data]
+            if len(present) > 1:
+                names = ", ".join(repr(name) for name in present)
                 raise ValueError(
-                    f"removed field(s) {names}; use 'additional_info' instead"
+                    f"metadata fields {names} cannot be used together; provide exactly one"
                 )
         return data
 
+    @property
+    def taskInstruction(self) -> Optional[Dict[str, Any]]:
+        """Legacy SDK attribute retained for callers of older CLI releases."""
+        return self.additionalInfo
+
     def relpath(self) -> str:
         """The local landing path relative to out (POSIX, normalized)."""
         return resolve_relpath(self.relativePath, self.originalPath, self.objectKey)

{dexverse_cli-0.0.4a1 → dexverse_cli-0.0.4.post2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "dexverse-cli"
-version = "0.0.4a1"
+version = "0.0.4.post2"
 description = "DexVerse dataset downloader for S3-compatible robot training datasets"
 readme = "README.md"
 requires-python = ">=3.9"

{dexverse_cli-0.0.4a1 → dexverse_cli-0.0.4.post2}/tests/test_convert_additional_info.py

@@ -20,8 +20,8 @@ def _write_minimal_hdf5(path: Path) -> None:
         joints.attrs["columns"] = json.dumps(["joint"])
 
 
-def _write_hidden_additional_info(root: Path, relpath: str, info: dict) -> Path:
-    path = st.additional_info_path(root, relpath)
+def _write_hidden_task_instruction(root: Path, relpath: str, info: dict) -> Path:
+    path = st.task_instruction_path(root, relpath)
     path.parent.mkdir(parents=True, exist_ok=True)
     path.write_text(
         json.dumps(info, ensure_ascii=False, indent=2) + "\n",
@@ -30,21 +30,21 @@ def _write_hidden_additional_info(root: Path, relpath: str, info: dict) -> Path:
     return path
 
 
-def test_convert_single_file_writes_and_updates_raw_additional_info(tmp_path):
+def test_convert_single_file_writes_and_updates_raw_task_instruction(tmp_path):
     download_root = tmp_path / "download"
     source = download_root / "session.hdf5"
     restored = tmp_path / "restored" / "session"
     _write_minimal_hdf5(source)
-    info_path = _write_hidden_additional_info(
+    info_path = _write_hidden_task_instruction(
         download_root,
         "session.hdf5",
-        {"aaa": "中文", "number": 1.1111111111111112e+54},
+        {"aaa": "中文", "number": 1.1111111111111112e+54, "data_protocol": "W1"},
     )
 
     frames = convert_hdf5(source, restored)
 
     assert frames == 1
-    output_path = restored / "additional_info.json"
+    output_path = restored / "task_instruction.json"
     assert json.loads(output_path.read_text(encoding="utf-8")) == {
         "aaa": "中文",
         "number": 1.1111111111111112e+54,
@@ -58,19 +58,19 @@ def test_convert_single_file_writes_and_updates_raw_additional_info(tmp_path):
     assert not output_path.exists()
 
 
-def test_convert_batch_preserves_paths_for_additional_info(tmp_path):
+def test_convert_batch_preserves_paths_for_task_instruction(tmp_path):
     download_root = tmp_path / "download"
     source = download_root / "nested" / "batch.hdf5"
     restored_root = tmp_path / "restored"
     _write_minimal_hdf5(source)
     info = {"task": "batch"}
-    _write_hidden_additional_info(download_root, "nested/batch.hdf5", info)
+    _write_hidden_task_instruction(download_root, "nested/batch.hdf5", info)
 
     success, frames = convert_batch(download_root, restored_root)
 
     assert success == 1
     assert frames == 1
-    output_path = restored_root / "nested" / "batch" / "additional_info.json"
+    output_path = restored_root / "nested" / "batch" / "task_instruction.json"
     assert json.loads(output_path.read_text(encoding="utf-8")) == info
 
 
@@ -82,4 +82,32 @@ def test_standalone_hdf5_without_downloader_metadata_stays_compatible(tmp_path):
     frames = convert_hdf5(source, restored)
 
     assert frames == 1
-    assert not (restored / "additional_info.json").exists()
+    assert not (restored / "task_instruction.json").exists()
+
+
+def test_convert_bridges_additional_info_state_to_legacy_output(tmp_path):
+    download_root = tmp_path / "download"
+    source = download_root / "session.hdf5"
+    restored = tmp_path / "restored" / "session"
+    _write_minimal_hdf5(source)
+    bridge_path = st.additional_info_path(download_root, "session.hdf5")
+    bridge_path.parent.mkdir(parents=True)
+    bridge_path.write_text('{"from": "32ffa8e", "data_protocol": "SIM"}\n', encoding="utf-8")
+
+    convert_hdf5(source, restored)
+
+    assert json.loads((restored / "task_instruction.json").read_text(encoding="utf-8")) == {
+        "from": "32ffa8e"
+    }
+
+
+def test_convert_data_protocol_only_writes_empty_task_instruction(tmp_path):
+    download_root = tmp_path / "download"
+    source = download_root / "session.hdf5"
+    restored = tmp_path / "restored" / "session"
+    _write_minimal_hdf5(source)
+    _write_hidden_task_instruction(download_root, "session.hdf5", {"data_protocol": "UMI"})
+
+    convert_hdf5(source, restored)
+
+    assert json.loads((restored / "task_instruction.json").read_text(encoding="utf-8")) == {}

{dexverse_cli-0.0.4a1 → dexverse_cli-0.0.4.post2}/tests/test_http_backend.py

@@ -273,7 +273,11 @@ def test_duplicate_filename_download_and_additional_info_keep_first(tmp_path, ht
         {
             "download_url": http_server.url("first.hdf5"),
             "file_name": "same.hdf5",
-            "additional_info": {"source": "第一条", "value": 1.1111111111111112e+54},
+            "additional_info": {
+                "source": "第一条",
+                "value": 1.1111111111111112e+54,
+                "data_protocol": "W1",
+            },
         },
         {
             "download_url": http_server.url("second.hdf5"),
@@ -290,8 +294,51 @@ def test_duplicate_filename_download_and_additional_info_keep_first(tmp_path, ht
     assert report.total == 1
     assert report.downloaded == 1
     assert (out / "same.hdf5").read_bytes() == first_body
-    info_path = st.additional_info_path(out, "same.hdf5")
-    assert json.loads(info_path.read_text(encoding="utf-8")) == payload[0]["additional_info"]
+    instruction_path = st.task_instruction_path(out, "same.hdf5")
+    assert json.loads(instruction_path.read_text(encoding="utf-8")) == {
+        "source": "第一条",
+        "value": 1.1111111111111112e+54,
+    }
+    assert manifest.files[0].additionalInfo["data_protocol"] == "W1"
+    assert not st.additional_info_path(out, "same.hdf5").exists()
+
+
+def test_legacy_manifest_metadata_still_syncs_to_legacy_path(tmp_path, http_server):
+    body = b"already-downloaded"
+    out = tmp_path / "out"
+    out.mkdir()
+    (out / "ep.hdf5").write_bytes(body)
+    manifest = Manifest.from_payload([{
+        "download_url": http_server.url("unused.hdf5"),
+        "file_name": "ep.hdf5",
+        "file_size": len(body),
+        "task_instruction": {"mode": "legacy", "data_protocol": "SIM"},
+    }])
+
+    report = DownloadManager(out, backend="http").run(manifest)
+
+    assert report.ok
+    instruction_path = st.task_instruction_path(out, "ep.hdf5")
+    assert json.loads(instruction_path.read_text(encoding="utf-8")) == {"mode": "legacy"}
+    assert not st.additional_info_path(out, "ep.hdf5").exists()
+
+
+def test_data_protocol_only_writes_an_empty_task_instruction(tmp_path, http_server):
+    body = b"already-downloaded"
+    out = tmp_path / "out"
+    out.mkdir()
+    (out / "ep.hdf5").write_bytes(body)
+    manifest = Manifest.from_payload([{
+        "download_url": http_server.url("unused.hdf5"),
+        "file_name": "ep.hdf5",
+        "file_size": len(body),
+        "additional_info": {"data_protocol": "UMI"},
+    }])
+
+    report = DownloadManager(out, backend="http").run(manifest)
+
+    assert report.ok
+    assert json.loads(st.task_instruction_path(out, "ep.hdf5").read_text(encoding="utf-8")) == {}
 
 
 def test_skipped_file_still_syncs_additional_info(tmp_path, http_server):
@@ -310,13 +357,13 @@ def test_skipped_file_still_syncs_additional_info(tmp_path, http_server):
 
     assert report.ok
     assert report.skipped == 1
-    info_path = st.additional_info_path(out, "ep.hdf5")
-    assert json.loads(info_path.read_text(encoding="utf-8")) == {"mode": "skip"}
+    instruction_path = st.task_instruction_path(out, "ep.hdf5")
+    assert json.loads(instruction_path.read_text(encoding="utf-8")) == {"mode": "skip"}
 
 
 def test_failed_or_corrupt_file_does_not_keep_additional_info(tmp_path, http_server):
     out = tmp_path / "out"
-    stale_path = st.additional_info_path(out, "ep.hdf5")
+    stale_path = st.task_instruction_path(out, "ep.hdf5")
     stale_path.parent.mkdir(parents=True)
     stale_path.write_text('{"stale": true}', encoding="utf-8")
     failed = Manifest.from_payload([{
@@ -350,7 +397,7 @@ def test_missing_additional_info_clears_stale_hidden_metadata(tmp_path, http_ser
     body = b"fresh-body"
     _seed(tmp_path, {"ep.hdf5": body})
     out = tmp_path / "out"
-    stale_path = st.additional_info_path(out, "ep.hdf5")
+    stale_path = st.task_instruction_path(out, "ep.hdf5")
     stale_path.parent.mkdir(parents=True)
     stale_path.write_text('{"stale": true}', encoding="utf-8")
     manifest = Manifest.from_payload([{

{dexverse_cli-0.0.4a1 → dexverse_cli-0.0.4.post2}/tests/test_manifest_aliases.py

@@ -48,25 +48,26 @@ def test_manifest_from_files_wrapper_native():
     assert m.files[0].presignedUrl.startswith("https")
 
 
-def test_additional_info_accepts_native_and_canonical_names():
+@pytest.mark.parametrize(
+    "metadata_field",
+    ["additional_info", "additionalInfo", "task_instruction", "taskInstruction"],
+)
+def test_metadata_accepts_new_and_legacy_names(metadata_field):
     info = {
         "aaa": "111",
         "aaaa": 222,
         "ewrwrwe": 1.1111111111111112e+54,
     }
-    native = Manifest.from_payload({
-        "files": [{**_native_file(), "additional_info": info}],
-    })
-    canonical = Manifest.from_payload({
-        "files": [{**_native_file(), "additionalInfo": info}],
+    manifest = Manifest.from_payload({
+        "files": [{**_native_file(), metadata_field: info}],
     })
 
-    assert native.files[0].additionalInfo == info
-    assert canonical.files[0].additionalInfo == info
-    for manifest in (native, canonical):
-        dumped_file = json.loads(manifest.model_dump_json())["files"][0]
-        assert dumped_file["additionalInfo"] == info
-        assert "additional_info" not in dumped_file
+    assert manifest.files[0].additionalInfo == info
+    assert manifest.files[0].taskInstruction == info
+    dumped_file = json.loads(manifest.model_dump_json())["files"][0]
+    assert dumped_file["additionalInfo"] == info
+    assert "additional_info" not in dumped_file
+    assert "task_instruction" not in dumped_file
 
 
 @pytest.mark.parametrize(
@@ -77,7 +78,7 @@ def test_additional_info_accepts_native_and_canonical_names():
     ],
     ids=["equal-values", "conflicting-values"],
 )
-def test_additional_info_aliases_cannot_be_used_together(
+def test_metadata_aliases_cannot_be_used_together(
     native_info,
     canonical_info,
 ):
@@ -91,18 +92,23 @@ def test_additional_info_aliases_cannot_be_used_together(
         })
 
 
-def test_additional_info_must_be_an_object():
+@pytest.mark.parametrize("metadata_field", ["additional_info", "task_instruction"])
+def test_metadata_must_be_an_object(metadata_field):
     with pytest.raises(ValidationError):
         Manifest.from_payload({
-            "files": [{**_native_file(), "additional_info": ["not", "an", "object"]}],
+            "files": [{**_native_file(), metadata_field: ["not", "an", "object"]}],
         })
 
 
 @pytest.mark.parametrize("legacy_field", ["task_instruction", "taskInstruction"])
-def test_removed_task_instruction_fields_are_rejected(legacy_field):
-    with pytest.raises(ValidationError, match="use 'additional_info' instead"):
+def test_new_and_legacy_metadata_fields_cannot_be_used_together(legacy_field):
+    with pytest.raises(ValidationError, match="cannot be used together"):
         Manifest.from_payload({
-            "files": [{**_native_file(), legacy_field: {"removed": True}}],
+            "files": [{
+                **_native_file(),
+                "additional_info": {"new": True},
+                legacy_field: {"legacy": True},
+            }],
         })