xnatbidscli 2.0.0__tar.gz

xnatbidscli-2.0.0/.claude/CLAUDE.md

@@ -0,0 +1 @@
+@../AGENTS.md

xnatbidscli-2.0.0/.github/workflows/publish.yml

@@ -0,0 +1,25 @@
+name: Publish to PyPI
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    permissions:
+      # Required for PyPI trusted publishing (OIDC) — no API token needed.
+      id-token: write
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          # hatch-vcs needs full history/tags to compute the version.
+          fetch-depth: 0
+
+      - uses: astral-sh/setup-uv@v5
+
+      - name: Build sdist and wheel
+        run: uv build
+
+      - name: Publish to PyPI
+        run: uv publish

xnatbidscli-2.0.0/.gitignore

@@ -0,0 +1,212 @@
+# specifics
+data/
+uv.lock
+biopac*
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[codz]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py.cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# UV
+#   Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#uv.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+#poetry.toml
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#   pdm recommends including project-wide configuration in pdm.toml, but excluding .pdm-python.
+#   https://pdm-project.org/en/latest/usage/project/#working-with-version-control
+#pdm.lock
+#pdm.toml
+.pdm-python
+.pdm-build/
+
+# pixi
+#   Similar to Pipfile.lock, it is generally recommended to include pixi.lock in version control.
+#pixi.lock
+#   Pixi creates a virtual environment in the .pixi directory, just like venv module creates one
+#   in the .venv directory. It is recommended not to include this directory in version control.
+.pixi
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.envrc
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+
+# Abstra
+# Abstra is an AI-powered process automation framework.
+# Ignore directories containing user credentials, local state, and settings.
+# Learn more at https://abstra.io/docs
+.abstra/
+
+# Visual Studio Code
+#  Visual Studio Code specific template is maintained in a separate VisualStudioCode.gitignore 
+#  that can be found at https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
+#  and can be added to the global gitignore or merged into this file. However, if you prefer, 
+#  you could uncomment the following to ignore the entire vscode folder
+# .vscode/
+
+# Ruff stuff:
+.ruff_cache/
+
+# PyPI configuration file
+.pypirc
+
+# Cursor
+#  Cursor is an AI-powered code editor. `.cursorignore` specifies files/directories to
+#  exclude from AI features like autocomplete and code analysis. Recommended for sensitive data
+#  refer to https://docs.cursor.com/context/ignore-files
+.cursorignore
+.cursorindexingignore
+
+# Marimo
+marimo/_static/
+marimo/_lsp/
+__marimo__/

xnatbidscli-2.0.0/.python-version

@@ -0,0 +1 @@
+3.11

xnatbidscli-2.0.0/.readthedocs.yaml

@@ -0,0 +1,15 @@
+version: 2
+
+build:
+  os: ubuntu-24.04
+  tools:
+    python: latest
+  jobs:
+    install:
+      - pip install -r docs/requirements.txt
+    build:
+      html:
+        - zensical build
+    post_build:
+      - mkdir -p $READTHEDOCS_OUTPUT/html/
+      - cp --recursive site/* $READTHEDOCS_OUTPUT/html/

xnatbidscli-2.0.0/AGENTS.md

@@ -0,0 +1,21 @@
+# AGENTS
+
+## Overview
+
+This project is a command-line interface for logging into an Extensible Neuroimaging Archive Toolkit (XNAT) server, querying experiments and downloading files, then converting to the Brain Imaging Data Structure (BIDS) standard format.
+
+## Always (for every prompt and follow-up)
+
+- Always ask clarifying questions if a request is unclear or if there are distinctly different performance or behavior expectations among possible solutions.
+- Always update the README.md and docs when something changes in the code that affects the behaviors described in the README.md or docs. But keep document changes concise so any given page does not end up too long.
+- Always make clear and concise comments and numpy docstrings inside the code. The code should be mostly self-explanatory, and comments should be used to clarify complex logic or design decisions.
+- Always use "uv run" to run anything requiring the environment in this repository.
+- Update the docs/faq.md if any functionality behaviors change.
+- Add every user-facing change under `[Unreleased]` in docs/changelog.md, following [Keep a Changelog](https://keepachangelog.com/en/2.0.0/).
+
+## Don't
+
+- Don't assume the user wants one option over another without explicitly asking a clarifying question.
+- Don't write code comments that are very verbose or redundant.
+- Don't ever enter, read, or touch a data/ directory. A data/ directory is for user data only and should never be read or modified by the agent.
+- Don't ever query, read, or touch the user's XNAT server. The user's XNAT server is for users only and should never be queried or used by the agent.

xnatbidscli-2.0.0/CHANGELOG.md

@@ -0,0 +1,3 @@
+# Changelog
+
+The changelog is maintained at [docs/changelog.md](docs/changelog.md) and published at <https://xnatbidscli.readthedocs.io/changelog/>.

xnatbidscli-2.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 NIMH Data Science and Sharing Team
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

xnatbidscli-2.0.0/PKG-INFO

@@ -0,0 +1,65 @@
+Metadata-Version: 2.5
+Name: xnatbidscli
+Version: 2.0.0
+Summary: CLI for logging into XNAT, downloading neuroimaging experiments, and converting them to BIDS.
+Project-URL: Homepage, https://github.com/nimh-dsst/xnat-bids-cli
+Project-URL: Documentation, https://xnatbidscli.readthedocs.io/
+Project-URL: Repository, https://github.com/nimh-dsst/xnat-bids-cli
+Project-URL: Issues, https://github.com/nimh-dsst/xnat-bids-cli/issues
+Author-email: Eric Earl <eric.earl@nih.gov>
+Maintainer-email: NIMH Data Science and Sharing Team <nimhdsst@mail.nih.gov>
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
+Requires-Python: >=3.11
+Requires-Dist: bioread>=3; python_version < '3.12'
+Requires-Dist: cubids>=1.2.1
+Requires-Dist: dcm2bids>=3
+Requires-Dist: dcm2niix>=1.0.20260416
+Requires-Dist: nibabel<5.4,>=5.3; python_version < '3.12'
+Requires-Dist: nibabel>=5.4.2; python_version >= '3.12'
+Requires-Dist: numpy<2
+Requires-Dist: pandas<2.4,>=2
+Requires-Dist: phys2bids>=2.10; python_version < '3.12'
+Requires-Dist: pydicom>=2.4
+Requires-Dist: pyxnat>=1.6
+Requires-Dist: requests>=2
+Description-Content-Type: text/markdown
+
+# XNAT CLI for BIDS
+
+`xnatbidscli` is a command-line toolkit that takes neuroimaging data from login to analysis-ready dataset: authenticate against an XNAT (Extensible Neuroimaging Archive Toolkit) server, query and download experiments, convert them to BIDS (Brain Imaging Data Structure) with `dcm2bids`, group acquisitions with CuBIDS, map participants/sessions to anonymized IDs, and fold in physiological recordings — all through one CLI built on [PyXNAT](https://pyxnat.github.io/pyxnat/index.html).
+
+## User Guide
+
+Full documentation for every `xnatbidscli` subcommand, the on-disk layouts each one produces, and how the pieces fit together lives at [xnatbidscli.readthedocs.io](https://xnatbidscli.readthedocs.io/).
+
+## Installation
+
+Requires Python ≥ 3.11.
+
+```bash
+pip install xnatbidscli
+```
+
+See the [Installation guide](https://xnatbidscli.readthedocs.io/installation/) for installing unreleased features from source with uv and the full list of runtime dependencies.
+
+## Reporting Issues and Feature Requests
+
+Use the [GitHub Issues](https://github.com/nimh-dsst/xnat-bids-cli/issues) feature here to report anything wrong with the CLI, code, or docs. You can also provide feature requests via GitHub Issues.
+
+## Attribution
+
+Developed and tested primarily by [Eric Earl](https://github.com/ericearl) of the [NIMH Data Science and Sharing Team](https://github.com/nimh-dsst) primarily for the NIMH Intramural Research Program labs. Claude Code and GitHub Copilot were used to develop features, docs, and modifications, as prompted by Eric Earl.
+
+## License
+
+Distributed under the MIT License.

xnatbidscli-2.0.0/README.md

@@ -0,0 +1,29 @@
+# XNAT CLI for BIDS
+
+`xnatbidscli` is a command-line toolkit that takes neuroimaging data from login to analysis-ready dataset: authenticate against an XNAT (Extensible Neuroimaging Archive Toolkit) server, query and download experiments, convert them to BIDS (Brain Imaging Data Structure) with `dcm2bids`, group acquisitions with CuBIDS, map participants/sessions to anonymized IDs, and fold in physiological recordings — all through one CLI built on [PyXNAT](https://pyxnat.github.io/pyxnat/index.html).
+
+## User Guide
+
+Full documentation for every `xnatbidscli` subcommand, the on-disk layouts each one produces, and how the pieces fit together lives at [xnatbidscli.readthedocs.io](https://xnatbidscli.readthedocs.io/).
+
+## Installation
+
+Requires Python ≥ 3.11.
+
+```bash
+pip install xnatbidscli
+```
+
+See the [Installation guide](https://xnatbidscli.readthedocs.io/installation/) for installing unreleased features from source with uv and the full list of runtime dependencies.
+
+## Reporting Issues and Feature Requests
+
+Use the [GitHub Issues](https://github.com/nimh-dsst/xnat-bids-cli/issues) feature here to report anything wrong with the CLI, code, or docs. You can also provide feature requests via GitHub Issues.
+
+## Attribution
+
+Developed and tested primarily by [Eric Earl](https://github.com/ericearl) of the [NIMH Data Science and Sharing Team](https://github.com/nimh-dsst) primarily for the NIMH Intramural Research Program labs. Claude Code and GitHub Copilot were used to develop features, docs, and modifications, as prompted by Eric Earl.
+
+## License
+
+Distributed under the MIT License.

xnatbidscli-2.0.0/docs/assets/js/faq.js

@@ -0,0 +1,12 @@
+document.addEventListener("DOMContentLoaded", () => {
+  const expandAll = document.getElementById("faq-expand-all");
+  const collapseAll = document.getElementById("faq-collapse-all");
+  const details = document.querySelectorAll("article details");
+
+  expandAll?.addEventListener("click", () => {
+    details.forEach((d) => (d.open = true));
+  });
+  collapseAll?.addEventListener("click", () => {
+    details.forEach((d) => (d.open = false));
+  });
+});

xnatbidscli-2.0.0/docs/changelog.md

@@ -0,0 +1,63 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/2.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [2.0.0] - 2026-09-28
+
+The project is renamed from `xnatcli` to `xnatbidscli` and is published on PyPI. Downloads are faster and more reliable, and you can now rename subjects and experiments between `query` and `download`.
+
+### Added
+
+- The package is set up for PyPI (`pip install xnatbidscli`). The version now comes from git tags, and a GitHub Action publishes each new GitHub release.
+- `download --accession ACCESSION` downloads by one subject ID or label, experiment ID or label, or StudyInstanceUID alone, with no project or subject needed. A subject downloads every experiment for that subject. An experiment downloads only that experiment. A label that matches more than one subject or experiment on the server is an error that lists the matches.
+- `query` filter flags: `--accession` (subject ID or label, experiment ID or label, or StudyInstanceUID), `--date`, `--time`, `--scanner`, `--study`, `--site`, `--operator`, `--sex`, `--handedness` and `--age`. Different flags must all match, and any one value of a flag can match. Text filters match a case-insensitive substring. Experiments with a blank value for a filtered field are dropped.
+- `query` CSVs have new columns at the end: `EXPERIMENT_TIME`, `STUDY_DESCRIPTION`, `STUDY_UID`, `SCANNER_NAME`, `SCANNER_MANUFACTURER`, `SCANNER_MODEL`, `SCANNER_SERIAL`, `SOFTWARE_VERSION`, `FIELD_STRENGTH`, `SITE`, `OPERATOR`, `XNAT_GENDER`, `DICOM_SEX`, `HANDEDNESS` and `AGE`. `XNAT_GENDER` and `DICOM_SEX` both use `M`/`F`/`O` codes, and `--sex` keeps an experiment if either one matches. `AGE`, `EXPERIMENT_TIME`, `STUDY_UID`, `SCANNER_NAME`, `SCANNER_MANUFACTURER`, `SCANNER_MODEL`, `SITE` and `OPERATOR` come from the DICOM header first, with XNAT's value as the fallback.
+- `download --rename-subject` and `--rename-experiment` rename the on-disk `SUBJECT`/`EXPERIMENT` directories in `-1` and `--accession` modes.
+- `query` CSVs have new columns: `SUBJECT_BIDS_RENAME`, `EXPERIMENT_BIDS_RENAME` and `ESTIMATED_SIZE_BYTES`. `download --csv` renames directories using the two rename columns. `ESTIMATED_SIZE_BYTES` is `UNKNOWN`, never `0` or blank, when no size can be determined.
+- `download` shows the bytes downloaded so far for each experiment in `--csv` mode and for a subject `--accession`.
+- `mriconvert` prints an elapsed-time line every 5 seconds while `dcm2bids` runs.
+- `query` shows a status line while it runs, updated every 5 seconds, with the elapsed time and the number of experiments checked and matched. It only appears when stdout is a terminal.
+- `mriconvert_qc.json` records a `LastModified` timestamp for `Dcm2BidsConfigPath`.
+- `mriconvert` backs up an existing `mriconvert_qc.tsv` to `OUTPUT_DIR/mriconvert_qc_backups/PROJECT-<P>_mriconvert_qc_<YYYYMMDD_HHMMSS>.tsv` before overwriting it with different content.
+- `physioconvert` has a new `SKIPPED` status. It skips an association when its output `_physio.tsv.gz` already exists and is valid.
+- Saved credentials at `~/.xnatcli/credentials.cfg` move automatically to `~/.xnatbidscli/credentials.cfg` the first time a command needs them.
+- New docs pages: [Manual Steps](manual.md), [Frequently Asked Questions](faq.md) and [Using Excel](excel.md). The [`query`](cli/query.md), [Manual Steps](manual.md) and [Quickstart](quickstart.md) pages now warn that the query CSV can contain PII, and that unrenamed labels carry it into `download` folder names and logs. There is also a "Project feedback" section on the [Overview](index.md) and a GitHub repository link on every page.
+
+### Changed
+
+- **Breaking:** The CLI command, Python package and config directory are renamed from `xnatcli` to `xnatbidscli`. Run `pip install xnatbidscli` (or re-run `uv sync` from source) to install the new command. See [Installation](installation.md#upgrading-from-xnatcli).
+- **Breaking:** `physioconvert` needs Python 3.11. The latest `phys2bids` release caps `numpy` below 1.24, which has no builds for Python 3.12 or newer. On Python 3.11, `xnatbidscli` installs everything, including `phys2bids`, and holds `numpy` at 1.23 and `nibabel` below 5.4. On Python 3.12 or newer, `phys2bids` and `bioread` are left out, every other subcommand works, and `physioconvert` exits with a message. See [Installation](installation.md).
+- **Breaking:** `download` now fetches each experiment as whole-experiment zip archives, not one file at a time. Files are saved using XNAT's own scan and resource folder names. The `PARTIAL` status is removed.
+- **Breaking:** `download -n/--ndownload` now sets how many experiments download in parallel. It is not used with `-1`.
+- **Breaking:** `query` output filenames now end in a `_<YYYYMMDD_HHMMSS>` timestamp, so a new run no longer overwrites an old file.
+- **Breaking:** The `query` CSV has a new column order: `PROJECT,SUBJECT_LABEL,SUBJECT_ID,SUBJECT_BIDS_RENAME,EXPERIMENT_LABEL,EXPERIMENT_ID,EXPERIMENT_DATE,EXPERIMENT_BIDS_RENAME,ESTIMATED_SIZE_BYTES`, followed by the new metadata columns listed above.
+- **Breaking:** The log CSVs from `download`, `mriconfig`, `mriconvert`, `physioconvert` and `cubids` have a new `USER` column after `DATESTAMP`.
+- `query` creates its CSV with mode `rw-------` (owner only) on Linux and macOS, since it can hold PII such as MRNs, scan dates and ages. It then prints an `INFO:` line explaining that sharing the CSV requires changing its group and permissions.
+- `query` now finishes with `Took N second(s) to write N row(s) to ...` instead of `Wrote N row(s) to ...`.
+- `query` makes one REST call per subject and three per experiment (record, DICOM header, size), so it is slower on large projects. Filters skip the later calls for experiments they drop.
+- `mriconvert` uses `sub-<label>`/`ses-<label>` directory names exactly as they are when `download` has already renamed them.
+- `download` Ctrl+C with `-n` > 1: experiments that have not started are cancelled, and the command exits right away.
+
+### Fixed
+
+- `download` is more stable, especially for session-level resources. It saves a single-file resource correctly when XNAT sends the file without a zip.
+- `download` now reports a dropped connection clearly, instead of showing a generic error.
+- When `physioconvert` runs in parallel, lines from different workers no longer get mixed together in the log.
+
+## [1.0.0] - 2026-07-15
+
+First release.
+
+### Added
+
+- `xnatcli` subcommands: `login`, `query`, `download`, `mriconfig`, `mriconvert`, `physioconvert`, `bidsmap` and `cubids`.
+- A documentation site built with Zensical and hosted on Read the Docs.
+
+[Unreleased]: https://github.com/nimh-dsst/xnat-bids-cli/compare/v2.0.0...HEAD
+[2.0.0]: https://github.com/nimh-dsst/xnat-bids-cli/compare/v1.0.0...v2.0.0
+[1.0.0]: https://github.com/nimh-dsst/xnat-bids-cli/releases/tag/v1.0.0

xnatbidscli-2.0.0/docs/cli/bidsmap.md

@@ -0,0 +1,51 @@
+# `xnatbidscli bidsmap`
+
+Generates a participant/session mapping TSV for a BIDS dataset — the output of `xnatbidscli mriconvert` (with any physio already placed by [`xnatbidscli physioconvert`](physioconvert.md)) — at `INPUT_DIR/PROJECT/`. The map is later filled in by hand to relate XNAT IDs and real dates to anonymized BIDS IDs and session codenames. When `-o OUTPUT_DIR` is provided, it additionally applies all filled-in renames by copying the BIDS dataset to a new directory tree.
+
+This is the "map" half of the xnatbidscli workflow: `mriconvert`/`physioconvert` first **convert** raw source data to BIDS, preserving the source data untouched; `bidsmap` then **maps** that raw/unmapped BIDS data to a separate, renamed BIDS output, so the intermediary unmapped BIDS data is preserved too. `bidsmap` operates on `.nii.gz` main files with `.json`/`.bval`/`.bvec` sidecars, reading its `rename` column and QC-exclusion columns (`recommend_for_use`, `complete`, `usable`, `qc_rating`) from `mriconvert_qc.tsv` (one row per `.nii.gz` file, so `rename` unambiguously targets one file). A physio `_physio.tsv.gz`/`.json` pair co-located under the same `sub-*/ses-*/<datatype>/` directory rides along the copy for free (participant/session label substitution only — [`physioconvert`](physioconvert.md) already writes it under its final name, so no separate rename step is needed for it).
+
+`mriconvert_qc.tsv` is named distinctly from BIDS's canonical `scans.tsv` (see [`xnatbidscli mriconvert`](mriconvert.md)); `bidsmap -o` promotes it to the canonical `scans.tsv`/`scans.json` in the mapped output.
+
+## Map TSV generation (always runs)
+
+1. Validates that `--input` exists and that the BIDS dataset `INPUT_DIR/PROJECT/` exists.
+2. Scans `INPUT_DIR/PROJECT/` for `sub-*` directories and, within each, `ses-*` subdirectories.
+3. Writes `INPUT_DIR/PROJECT-<PROJECT>_bidsmap.tsv` with the columns `participant_id`, `participant_rename`, `session_id`, `session_rename`. One row is emitted per `(participant, session)` pair, with the two `*_rename` columns left blank for later editing. Rows are sorted alphanumerically by `participant_id` then `session_id`.
+    - If the dataset has **no** sessions (no participant has any `ses-*` subdirectory), only the `participant_id` and `participant_rename` columns are written, one row per participant.
+    - If the dataset uses sessions but a particular participant has no `ses-*` subdirectory, that participant is skipped with a warning.
+4. If `PROJECT-<PROJECT>_bidsmap.tsv` already exists, a fresh blank map is generated and compared to it (with `pandas`): any `(participant_id, session_id)` pairs not already present are appended, and all existing rows — including any `*_rename` values already filled in — are preserved. The merged table is re-sorted and rewritten.
+
+```bash
+xnatbidscli bidsmap -i MRICONVERT_OUTPUT_DIR -p PROJECT
+```
+
+## Copy-with-rename (`-o OUTPUT_DIR`)
+
+When `-o` is provided, after updating the map TSV the command reads back the renames and writes a fully renamed copy of the BIDS dataset to `OUTPUT_DIR/PROJECT/`:
+
+- **`PROJECT-<PROJECT>_bidsmap.tsv`** — `participant_rename` and `session_rename` columns rename `sub-*` and `ses-*` directory names and the matching labels embedded in all filenames. Blank values mean "keep the original label."
+- **`mriconvert_qc.tsv`** — its `rename` column supplies a corrected `bids_name` (the part after `sub-X[_ses-Y]_`) for the file(s) that row describes. Sidecar files (`.json`, `.bval`, `.bvec`) sharing the same stem are renamed to match. Blank values mean "keep the original bids_name."
+
+The copy also:
+
+- **QC filtering**: Rows whose `recommend_for_use`, `complete`, or `usable` is exactly `"FALSE"`, or whose `qc_rating` is exactly `"FAIL"` or `"UNCERTAIN"`, have their file(s) excluded from the copy (along with sidecars) and their row omitted from the output manifest. Values in any of these columns that are non-empty but do not match a valid Level from `mriconvert_qc.json` (e.g. `"false"` instead of `"FALSE"`) generate an additional warning, since they are silently ignored by the filter.
+- Updates `participants.tsv` in the output with the renamed participant IDs, and writes `scans.tsv` (`filename`, `bids_name`, `participant_id`, `session_id` columns, among others) to reflect all renames, omits rows for QC-excluded files, and drops the columns `rename` and `physio` (both have already been applied by the time `bidsmap -o` runs — `rename` to `bids_name`, `physio` by `xnatbidscli physioconvert`, which must run before `bidsmap -o`). All other reviewer columns are preserved. `mriconvert_qc.tsv`/`mriconvert_qc.json` and `physioconvert_qc.tsv`/`.json` themselves are **not** copied — `mriconvert_qc.tsv`/`.json` are promoted to `scans.tsv`/`.json` instead, and `physioconvert_qc.tsv`/`.json` stay raw-tree-only bookkeeping with no promoted counterpart.
+- Skips `tmp_dcm2bids` and `log` scratch directories.
+- **Incremental by default**: if `OUTPUT_DIR/PROJECT/` already exists, files under `sub-*/` whose destination path already exists are treated as already mapped and left untouched — only files not yet present at the destination are copied. Root-level manifests (`scans.tsv`, `participants.tsv`, `dataset_description.json`, ...) are always re-copied/re-written and re-patched, since they reflect the fully merged source state. This lets `bidsmap -o` be re-run safely as new data lands in `INPUT_DIR/PROJECT/` (e.g. from further `mriconvert`/`physioconvert` runs).
+- **Warns loudly** when new files are being mapped into a `sub-*/[ses-*/]` directory that already existed at the destination before this run, since that session was already mapped and is only gaining files.
+- Warns loudly for any two source files that would map to the same destination path (neither is copied); all warnings are re-displayed together at the end.
+
+```bash
+xnatbidscli bidsmap -i MRICONVERT_OUTPUT_DIR -p PROJECT -o RENAMED_OUTPUT_DIR
+```
+
+For example, if the BIDS dataset lives at `/data/bids/MYPROJ/`, then:
+
+- `xnatbidscli bidsmap -i /data/bids -p MYPROJ` writes `/data/bids/PROJECT-MYPROJ_bidsmap.tsv`.
+- After filling in the rename columns, `xnatbidscli bidsmap -i /data/bids -p MYPROJ -o /data/renamed` copies the dataset to `/data/renamed/MYPROJ/` with all renames applied.
+
+| Argument | Description |
+| --- | --- |
+| `-i`, `--input` | **Required.** Root directory holding the BIDS dataset at `INPUT_DIR/PROJECT/`. The map TSV is written here as `PROJECT-<PROJECT>_bidsmap.tsv`. |
+| `-p`, `--project` | **Required.** Project directory name under `INPUT_DIR` identifying the BIDS dataset to scan. |
+| `-o`, `--output` | *Optional.* When provided, copy the BIDS dataset to `OUTPUT_DIR/PROJECT/` with all renames from the map TSV and `mriconvert_qc.tsv`'s own `rename` column applied. If `OUTPUT_DIR/PROJECT/` already exists, only files not already mapped there are copied (see above). |

xnatbidscli-2.0.0/docs/cli/cubids.md

@@ -0,0 +1,24 @@
+# `xnatbidscli cubids`
+
+Runs [`CuBIDS`](https://cubids.readthedocs.io/) on a BIDS dataset produced by `xnatbidscli mriconvert` — first `cubids add-nifti-info` (which annotates JSON sidecars with NIfTI header fields) and then `cubids group` (which groups acquisitions by their parameters and writes `_summary.tsv`, `_files.tsv`, `_AcqGrouping.tsv`, and `_AcqGroupInfo.txt`).
+
+1. Validates that `--input` is an existing directory and that `INPUT_DIR/<PROJECT>/` (the BIDS dataset) exists. The expected layout is the one produced by `xnatbidscli mriconvert` — `<input>/PROJECT/sub-X/ses-Y/...`.
+2. Verifies that `cubids` is on `PATH`; exits with an error if it is missing.
+3. Creates the output directory `INPUT_DIR/PROJECT-<PROJECT>_cubids/` if it does not already exist. If it does, the directory is reused (CuBIDS writes its own outputs into it).
+4. If `INPUT_DIR/<PROJECT>/tmp_dcm2bids/` exists (leftover dcm2bids scratch), it is moved out to `INPUT_DIR/.<PROJECT>_cubids_stash_tmp_dcm2bids/` for the duration of the run so CuBIDS does not scan it, and moved back when the run finishes (success or failure). CuBIDS has no built-in ignore mechanism; it walks the whole BIDS tree.
+5. Invokes `cubids add-nifti-info <bids_dir>` without `--use-datalad` (datalad is disabled by default in CuBIDS), so the BIDS dataset itself is mutated in place to add NIfTI header info to sidecars.
+6. Invokes `cubids group <bids_dir> v0`, which writes `v0_summary.tsv`, `v0_files.tsv`, `v0_AcqGrouping.tsv`, and `v0_AcqGroupInfo.txt` into `INPUT_DIR/<PROJECT>/code/CuBIDS/`.
+7. On a successful `group`, the `INPUT_DIR/<PROJECT>/code/CuBIDS/` directory is merged into `INPUT_DIR/PROJECT-<PROJECT>_cubids/CuBIDS/` (existing files with the same name are overwritten; unrelated files in the destination are left alone) and the source is removed. If `INPUT_DIR/<PROJECT>/code/` is empty afterwards, it is also removed.
+8. If `add-nifti-info` exits non-zero, `group` is skipped and the command exits `1`. Otherwise the exit code is `0` if both steps succeeded and `1` if `group` failed.
+
+```bash
+xnatbidscli cubids -i BIDSCONVERT_OUTPUT_DIR -p PROJECT
+```
+
+For example, if you ran `xnatbidscli mriconvert -i DOWNLOAD_DIR -p MYPROJ -o /data/bids -c config.json`, the BIDS dataset lives at `/data/bids/MYPROJ/`, and `xnatbidscli cubids -i /data/bids -p MYPROJ` writes CuBIDS outputs to `/data/bids/PROJECT-MYPROJ_cubids/CuBIDS/v0_*.tsv`.
+
+| Argument | Description |
+| --- | --- |
+| `-i`, `--input` | **Required.** Parent directory holding the BIDS dataset at `INPUT_DIR/PROJECT/` (i.e., the output of `xnatbidscli mriconvert`). The CuBIDS output subdirectory is created here. |
+| `-p`, `--project` | **Required.** Project directory name under `INPUT_DIR` identifying the BIDS dataset to process. |
+| `-l`, `--log` | *Optional.* Write a per-step log CSV to `INPUT_DIR/PROJECT-<PROJECT>_cubids/log/cubids_<YYYYMMDD_HHMMSS>_log.csv` with header `DATESTAMP,USER,PROJECT,STEP,STATUS`. One row per CuBIDS step (`add-nifti-info`, `group`), each with `STATUS` of `COMPLETE` or `FAILURE`. |