datalad-worktree 0.4.0__tar.gz

datalad_worktree-0.4.0/.githooks/pre-commit

@@ -0,0 +1,25 @@
+#!/bin/sh
+# Run the same lint CI runs, before the commit rather than after the push.
+#
+# Enable with:  git config core.hooksPath .githooks
+#
+# NOTE: git hooks do not fire for `jj commit` -- jujutsu runs no hooks at all.
+# If you commit with jj, use `jj lint` / `jj fix` instead; see CONTRIBUTING.md.
+
+set -e
+
+if ! command -v uv >/dev/null 2>&1; then
+    echo "pre-commit: uv not on PATH, skipping ruff" >&2
+    exit 0
+fi
+
+# Whole tree, not just staged files: ruff on this repo takes well under a
+# second, and a per-file check would miss an unused import left behind in a
+# file this commit does not touch.
+if ! uv run --dev ruff check .; then
+    echo >&2
+    echo "pre-commit: ruff found problems (this is what CI's lint job checks)." >&2
+    echo "  fix them:        uv run --dev ruff check --fix ." >&2
+    echo "  commit anyway:   git commit --no-verify" >&2
+    exit 1
+fi

datalad_worktree-0.4.0/.github/workflows/publish.yml

@@ -0,0 +1,33 @@
+name: Publish to PyPI
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+      - uses: astral-sh/setup-uv@v10.1.0
+      - name: Build package
+        run: uv build
+      - uses: actions/upload-artifact@v7
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    name: Upload release to PyPI
+    needs: build
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write
+    steps:
+      - uses: actions/download-artifact@v8
+        with:
+          name: dist
+          path: dist/
+      - name: Publish package distributions to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

datalad_worktree-0.4.0/.github/workflows/tests.yml

@@ -0,0 +1,44 @@
+name: Tests
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11", "3.12", "3.13"]
+    steps:
+      - uses: actions/checkout@v7
+
+      - name: Install git-annex
+        run: sudo apt-get update && sudo apt-get install -y git-annex
+
+      - name: Configure git identity for tests
+        run: |
+          git config --global user.email "test@example.com"
+          git config --global user.name "Test User"
+
+      - uses: astral-sh/setup-uv@v10.1.0
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install dependencies
+        run: uv sync --dev --locked
+
+      - name: Run tests
+        run: uv run pytest -v
+
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+      - uses: astral-sh/setup-uv@v10.1.0
+      - name: Install dependencies
+        run: uv sync --dev --locked
+      - name: Run ruff
+        run: uv run ruff check .

datalad_worktree-0.4.0/.gitignore

@@ -0,0 +1,35 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+
+# Distribution / packaging
+dist/
+build/
+*.egg-info/
+*.egg
+
+# Virtual environments
+.venv/
+venv/
+env/
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+*~
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Testing
+.pytest_cache/
+.coverage
+htmlcov/
+
+# uv
+.python-version

datalad_worktree-0.4.0/CHANGELOG.md

@@ -0,0 +1,39 @@
+# Changelog
+
+User-facing changes per release. Behaviour is documented in [docs/cli.md](docs/cli.md).
+
+## 0.4.0 (2026-10-05)
+
+### Breaking
+
+- `remove` is renamed `delete`; `worktree` alone runs `list`.
+- `add` takes `<branch> <worktree-path>`, in that order.
+- `add` replaces an existing worktree by default; `-f` now only discards commits the main checkout lacks.
+- `add` always starts the branch at each dataset's HEAD; `--no-create-branch` is gone.
+- `delete` deletes the branch by default (`--keep-branch` opts out), discards uncommitted changes, and no longer prompts; `--delete-branch` and `-y` are gone.
+- `--no-color` is gone; colour is off when output is not a terminal.
+- Python API: `create_nested_worktrees(replace=)` is gone.
+
+### Added
+
+- `worktree fetch`: bring a worktree's results home, or new code and inputs into it.
+- `add --follow-parent [<commit>]`: check subdatasets out at the commits their parent records.
+- `add` configures container bind mounts (`--no-bindpaths` skips).
+- `add` copies mtimes from the main checkout, Snakemake markers included (`--no-mtimes` skips).
+- `delete -n`.
+
+### Changed
+
+- `add` and `delete` are all-or-nothing: every check runs first, and `-n` runs the same checks.
+- `add` rolls back the worktrees it created if git fails partway.
+- `list` groups by hierarchy and prunes worktrees deleted by other means.
+- Pastel report colours, `delete` in pink.
+
+### Fixed
+
+- The main working tree is never deleted.
+- Works with current DataLad (no deprecated `eval_results` import).
+
+## 0.3.0 and earlier
+
+See the git history.

datalad_worktree-0.4.0/CLAUDE.md

@@ -0,0 +1,47 @@
+# CLAUDE.md
+
+Guidance for coding agents working on this repository. This file holds rules and pointers, not explanations. Each fact lives in exactly one place, so read it there:
+
+| What | Where |
+|---|---|
+| What the tool is for, install | [README.md](README.md) |
+| What each command and flag does | [docs/cli.md](docs/cli.md) |
+| **Why**, across commands: the premise, containers, mtimes | [docs/design.md](docs/design.md) |
+| **Why** a command or module is shaped the way it is, with the evidence | its module and function docstrings |
+| Setup, test and lint commands, jj hooks | [CONTRIBUTING.md](CONTRIBUTING.md) |
+| What changed in each release | [CHANGELOG.md](CHANGELOG.md) |
+
+## Before changing behaviour
+
+- **Read design.md and the docstrings of the code you are changing first.** Much of the code looks simplifiable and isn't: the obvious alternative was usually tried and broke something real, and the docstring next to it records what. If your change contradicts design.md or such a docstring, raise that with the maintainer instead of quietly working around it.
+- **Some guards are pinned by tests that fail when the guard is removed.** Don't weaken these to get a test passing:
+  - the `mtimes.py` safety rules and their `tests/test_mtimes.py` guards;
+  - `_worktree_kind()` in `delete.py` and its tests;
+  - `test_fails_without_bindpaths`.
+- **Ask for a design decision instead of picking one.** Changing a default, adding a flag, or changing what gets refused is the maintainer's call. Propose the options and trade-offs, then wait.
+
+## Making a change
+
+- **User-facing changes reach both front-ends.** `cli.py` (argparse) and `dl_command.py` (the DataLad `Interface` classes) each declare every flag. A new outcome is a `WorktreeResult`, rendered in `cli.py:_render_report()` and mapped in `dl_command.py`.
+- **All git access goes through `subprocess.run(..., capture_output=True, text=True)`.** No gitpython, and no runtime dependencies beyond the standard library. DataLad stays optional.
+- **Resolve datasets against each other with `is_git_repo_root()`, not `is_git_repo()`.** The latter walks up the tree and is true at an empty submodule mount point.
+- **Style:** `from __future__ import annotations`, full type hints, `logger = logging.getLogger(__name__)` per module. `uv run --dev ruff check .` must be clean.
+
+## Tests
+
+- **Use real repositories, never mocked git.** Tests build DataLad hierarchies in `tmp_path`, so `datalad` is a dev dependency. The full suite takes about 7 minutes and needs Linux. While iterating, run only the affected module.
+- **Every bug fix gets a regression test that fails without the fix.** Show it: revert the fix, watch the test fail, restore it, and say in the PR that you did.
+- **Assert invariants, not pinned values.** That means orderings, equality between trees, and paths left untouched. The generators also yield container and mtime reports, so filter reports by result type instead of counting all of them.
+- **Stamp and check mtimes with `follow_symlinks=False`.** Annexed files are symlinks, so the default touches the shared annex object instead and silently makes staleness tests vacuous.
+- **Some tests need extra capabilities.** `tests/test_containers_run.py` needs `datalad-container` and unprivileged user and mount namespaces, and skips itself without them. Don't turn those skips into failures.
+
+## Docs
+
+- **Update docs in the same commit as the change:** behaviour in `docs/cli.md`, rationale with its evidence in the docstring of the code it explains, or in `docs/design.md` when it spans commands, and README only when a headline claim changes.
+- **Never put rationale in this file.** Copies drift, so point to where the explanation lives instead.
+
+## Commits and PRs
+
+- **One concern per commit.** A refactor, a fix, a dependency change and a doc edit are separate commits.
+- **Commit subjects use a prefix:** `BF:` (fix), `ENH:` (feature), `DOC:`, `CI:`, `NF:`, or `API:`. The body explains why, and references the issue (`Closes #N`).
+- **History is linear.** The maintainer uses jj and stacks PRs; rebase, and never merge `master` into a branch.

datalad_worktree-0.4.0/CONTRIBUTING.md

@@ -0,0 +1,51 @@
+# Contributing
+
+Thanks for considering a contribution to `datalad-worktree`.
+
+## Setup
+
+```bash
+git clone https://github.com/just-meng/datalad-worktree.git
+cd datalad-worktree
+uv sync --dev
+```
+
+## Running tests and lint
+
+```bash
+uv run pytest
+uv run ruff check .
+```
+
+Worktrees of annexed repos and `datalad containers-run` don't work on Windows, so the full test suite needs Linux (or WSL). `tests/test_containers_run.py` additionally needs unprivileged user/mount namespaces and skips itself where those aren't available.
+
+### Running lint before you commit
+
+CI's `lint` job is `uv run ruff check .`, and finding out from CI costs a round trip. A hook that runs it locally lives in `.githooks/`; enable it with:
+
+```bash
+git config core.hooksPath .githooks
+```
+
+It checks the whole tree (ruff takes well under a second here), and `git commit --no-verify` bypasses it.
+
+**If you commit with [jujutsu](https://jj-vcs.github.io/), that hook never fires** — jj runs no hooks at all, `pre-commit` included, and there is no hook mechanism in its config. Two jj-native equivalents; both are repo-local config, so they have to be set per clone:
+
+```bash
+# `jj lint` -- exactly what CI runs
+jj config set --repo aliases.lint \
+  '["util", "exec", "--", "uv", "run", "--dev", "ruff", "check", "."]'
+
+# `jj fix` -- rewrite the commit with ruff's fixes applied
+jj config set --repo fix.tools.ruff.command \
+  '["uv", "run", "--dev", "ruff", "check", "--fix", "--quiet", "--stdin-filename=$path", "-"]'
+jj config set --repo fix.tools.ruff.patterns '["glob:**/*.py"]'
+```
+
+`jj fix` works because ruff reads a file on stdin and writes the fixed version to stdout, which is the interface `jj fix` expects. It rewrites the working-copy commit by default, or `jj fix -s <rev>` a particular one — handy when CI goes red on a commit in the middle of a stack, since descendants are rebased for you.
+
+## Submitting changes
+
+Open a pull request against `master`. Keep commits focused -- one logical change per commit, with a message explaining *why*, not just *what*. `uv run ruff check .` should pass clean.
+
+Why the code is shaped the way it is: [docs/design.md](docs/design.md) for what spans commands, and the docstrings of the code you are changing for the rest; read both before you change it. Coding agents: see [CLAUDE.md](CLAUDE.md).

datalad_worktree-0.4.0/PKG-INFO

@@ -0,0 +1,105 @@
+Metadata-Version: 2.5
+Name: datalad-worktree
+Version: 0.4.0
+Summary: Create nested git worktrees for DataLad datasets
+Author-email: Jiameng Wu <jiameng.wu@gmail.com>
+License-Expression: MIT
+Requires-Python: >=3.11
+Provides-Extra: datalad
+Requires-Dist: datalad; extra == 'datalad'
+Description-Content-Type: text/markdown
+
+# datalad-worktree
+
+Nested [git worktrees](https://git-scm.com/docs/git-worktree) for [DataLad](https://www.datalad.org/) dataset hierarchies: create one for a run, ship its results home, dispose of it — or simply repeat.
+
+## What it is for
+
+DataLad records provenance, Snakemake automates pipelines. Combining the two gives you [an automated computational workflow with staleness detection and full provenance](https://blog.datalad.org/posts/snakemake-datalad-worktree/). A long run then wants a checkout of its own, so that development can continue while it computes — which is exactly what `git worktree` is for.
+
+Except a DataLad superdataset is not one repository. It needs a worktree per dataset, wired together so submodule mount points line up, and two things break when you do that:
+
+- **`datalad containers-run` cannot read annexed files in a worktree.** `.git` is a link into the main repository, so annex object symlinks resolve outside the worktree, where the container cannot see them — `FileNotFoundError` on every annexed input ([datalad-container#288](https://github.com/datalad/datalad-container/issues/288)).
+- **A fresh worktree looks arbitrarily stale.** Git records content, not timestamps, so for a make-style pipeline the mtime *ordering* that **is** the up-to-date state is gone — and systematically inverted, since the superdataset is checked out before its subdatasets. Every `code/` file ends up newer than every output derived from it: "all your code changed". Bringing results back has the mirror problem — git moves only what it rewrites, so a run that reproduces byte-identical output produces no commit at all, nothing moves, and that output stays stale forever, rerunning on every invocation.
+
+`datalad-worktree` fixes both, in one command over the whole hierarchy:
+
+```bash
+cd /data/my-project
+worktree add runs /tmp/worktrees/runs   # one worktree per dataset, mtimes preserved,
+                                        # containers configured to reach the annex
+
+cd /tmp/worktrees/runs
+snakemake code/Snakefile -c 1 -k        # the long run — keep developing in main meanwhile
+
+cd /data/my-project
+worktree fetch runs                     # results home, mtimes included
+worktree delete runs                    # dispose of it — or overwrite with `worktree add`
+                                        # in the same location next time
+```
+
+`add` creates a worktree for the superdataset and every *installed* subdataset, nested the same way; uninstalled ones are skipped:
+
+```
+/tmp/worktrees/runs/        <- superdataset worktree (branch: runs)
+├── inputs/                 <- subdataset worktree
+├── code/                   <- subdataset worktree
+└── results/
+```
+
+Each is on branch `runs`, starting from that dataset's current state, even if `runs` already exists.
+
+`fetch` brings the worktree's commits back into each dataset's main checkout, then copies their mtimes across. `delete` removes the worktree from every dataset along with its branch, and refuses when the worktree contains unfetched commits.
+
+## Highlights
+
+- **mtimes are preserved in both directions** — creating a worktree and fetching from one — so staleness detection keeps working and finished work is not recomputed. A file inherits a timestamp only when its content is identical on both sides, matched by git's content hash rather than by filename. Snakemake's untracked `.snakemake_timestamp` markers travel too, so an output marked up to date with `snakemake --touch` stays up to date.
+- **`datalad containers-run` works inside the worktree**, via bind-mount configuration written per worktree, so the machine-specific paths stay out of the main checkout and out of history. One empty placeholder is committed on the worktree branch, which is what keeps a run record made there rerunnable elsewhere.
+- **Reproduce any recorded state.** `worktree add [branch] [path] --follow-parent <commit-or-tag>` checks every dataset out exactly as that superdataset commit recorded it, including which subdatasets existed then — to rerun a `datalad run` record from a clean slate, say. Without a commit, subdatasets follow what the superdataset records now.
+- **Results ship home without a clean tree.** Where your side has no commits of its own, `fetch` fast-forwards: nothing is rewritten, so unrelated work in progress is left alone. Where both sides have moved, it merges. It also runs the other way: from inside a worktree, `worktree fetch` brings new code and inputs in.
+- **Safe by default.** Creation and deletion are all-or-nothing: if any dataset would fail, none is touched. Deletion never touches the main working tree or a directory git does not call a worktree; it refuses a worktree with unfetched commits, unless forced, and discards uncommitted changes, as `add` does. Every command that changes something takes `--dry-run`.
+- **No dependencies.** Python standard library plus `git`. DataLad itself is optional — it only adds the `datalad worktree-*` commands.
+
+## Installation
+
+As a DataLad extension:
+
+```bash
+uv tool install datalad \
+  --with datalad-worktree@git+https://github.com/just-meng/datalad-worktree.git
+```
+
+As a standalone CLI tool:
+
+```bash
+git clone https://github.com/just-meng/datalad-worktree.git
+cd datalad-worktree
+uv sync         # --dev for the test suite
+```
+
+## Commands
+
+```bash
+worktree add <branch> <worktree-path>   # create nested worktrees
+worktree fetch [target]                 # bring commits and mtimes into the checkout you stand in
+worktree delete <target>                # remove worktrees, deepest-first
+worktree list                           # show worktrees across the hierarchy (also the default)
+```
+
+Installed as a DataLad extension — that is, into the same environment as DataLad, as above — each one is also available as `datalad worktree-add`, `worktree-list`, `worktree-delete`, `worktree-fetch`. Installed standalone, only the `worktree` command exists.
+
+Flags and behaviour: [docs/cli.md](docs/cli.md). Why it works the way it does: [docs/design.md](docs/design.md).
+
+## Requirements
+
+- Python >= 3.11
+- git on PATH (>= 2.38 for `fetch` to merge diverged datasets rather than refuse)
+- [DataLad](https://www.datalad.org/), optional
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md).
+
+## License
+
+MIT

datalad_worktree-0.4.0/README.md

@@ -0,0 +1,94 @@
+# datalad-worktree
+
+Nested [git worktrees](https://git-scm.com/docs/git-worktree) for [DataLad](https://www.datalad.org/) dataset hierarchies: create one for a run, ship its results home, dispose of it — or simply repeat.
+
+## What it is for
+
+DataLad records provenance, Snakemake automates pipelines. Combining the two gives you [an automated computational workflow with staleness detection and full provenance](https://blog.datalad.org/posts/snakemake-datalad-worktree/). A long run then wants a checkout of its own, so that development can continue while it computes — which is exactly what `git worktree` is for.
+
+Except a DataLad superdataset is not one repository. It needs a worktree per dataset, wired together so submodule mount points line up, and two things break when you do that:
+
+- **`datalad containers-run` cannot read annexed files in a worktree.** `.git` is a link into the main repository, so annex object symlinks resolve outside the worktree, where the container cannot see them — `FileNotFoundError` on every annexed input ([datalad-container#288](https://github.com/datalad/datalad-container/issues/288)).
+- **A fresh worktree looks arbitrarily stale.** Git records content, not timestamps, so for a make-style pipeline the mtime *ordering* that **is** the up-to-date state is gone — and systematically inverted, since the superdataset is checked out before its subdatasets. Every `code/` file ends up newer than every output derived from it: "all your code changed". Bringing results back has the mirror problem — git moves only what it rewrites, so a run that reproduces byte-identical output produces no commit at all, nothing moves, and that output stays stale forever, rerunning on every invocation.
+
+`datalad-worktree` fixes both, in one command over the whole hierarchy:
+
+```bash
+cd /data/my-project
+worktree add runs /tmp/worktrees/runs   # one worktree per dataset, mtimes preserved,
+                                        # containers configured to reach the annex
+
+cd /tmp/worktrees/runs
+snakemake code/Snakefile -c 1 -k        # the long run — keep developing in main meanwhile
+
+cd /data/my-project
+worktree fetch runs                     # results home, mtimes included
+worktree delete runs                    # dispose of it — or overwrite with `worktree add`
+                                        # in the same location next time
+```
+
+`add` creates a worktree for the superdataset and every *installed* subdataset, nested the same way; uninstalled ones are skipped:
+
+```
+/tmp/worktrees/runs/        <- superdataset worktree (branch: runs)
+├── inputs/                 <- subdataset worktree
+├── code/                   <- subdataset worktree
+└── results/
+```
+
+Each is on branch `runs`, starting from that dataset's current state, even if `runs` already exists.
+
+`fetch` brings the worktree's commits back into each dataset's main checkout, then copies their mtimes across. `delete` removes the worktree from every dataset along with its branch, and refuses when the worktree contains unfetched commits.
+
+## Highlights
+
+- **mtimes are preserved in both directions** — creating a worktree and fetching from one — so staleness detection keeps working and finished work is not recomputed. A file inherits a timestamp only when its content is identical on both sides, matched by git's content hash rather than by filename. Snakemake's untracked `.snakemake_timestamp` markers travel too, so an output marked up to date with `snakemake --touch` stays up to date.
+- **`datalad containers-run` works inside the worktree**, via bind-mount configuration written per worktree, so the machine-specific paths stay out of the main checkout and out of history. One empty placeholder is committed on the worktree branch, which is what keeps a run record made there rerunnable elsewhere.
+- **Reproduce any recorded state.** `worktree add [branch] [path] --follow-parent <commit-or-tag>` checks every dataset out exactly as that superdataset commit recorded it, including which subdatasets existed then — to rerun a `datalad run` record from a clean slate, say. Without a commit, subdatasets follow what the superdataset records now.
+- **Results ship home without a clean tree.** Where your side has no commits of its own, `fetch` fast-forwards: nothing is rewritten, so unrelated work in progress is left alone. Where both sides have moved, it merges. It also runs the other way: from inside a worktree, `worktree fetch` brings new code and inputs in.
+- **Safe by default.** Creation and deletion are all-or-nothing: if any dataset would fail, none is touched. Deletion never touches the main working tree or a directory git does not call a worktree; it refuses a worktree with unfetched commits, unless forced, and discards uncommitted changes, as `add` does. Every command that changes something takes `--dry-run`.
+- **No dependencies.** Python standard library plus `git`. DataLad itself is optional — it only adds the `datalad worktree-*` commands.
+
+## Installation
+
+As a DataLad extension:
+
+```bash
+uv tool install datalad \
+  --with datalad-worktree@git+https://github.com/just-meng/datalad-worktree.git
+```
+
+As a standalone CLI tool:
+
+```bash
+git clone https://github.com/just-meng/datalad-worktree.git
+cd datalad-worktree
+uv sync         # --dev for the test suite
+```
+
+## Commands
+
+```bash
+worktree add <branch> <worktree-path>   # create nested worktrees
+worktree fetch [target]                 # bring commits and mtimes into the checkout you stand in
+worktree delete <target>                # remove worktrees, deepest-first
+worktree list                           # show worktrees across the hierarchy (also the default)
+```
+
+Installed as a DataLad extension — that is, into the same environment as DataLad, as above — each one is also available as `datalad worktree-add`, `worktree-list`, `worktree-delete`, `worktree-fetch`. Installed standalone, only the `worktree` command exists.
+
+Flags and behaviour: [docs/cli.md](docs/cli.md). Why it works the way it does: [docs/design.md](docs/design.md).
+
+## Requirements
+
+- Python >= 3.11
+- git on PATH (>= 2.38 for `fetch` to merge diverged datasets rather than refuse)
+- [DataLad](https://www.datalad.org/), optional
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md).
+
+## License
+
+MIT

datalad_worktree-0.4.0/docs/cli.md

@@ -0,0 +1,147 @@
+# CLI reference
+
+```bash
+worktree add runs /tmp/worktrees/runs           # standalone CLI
+datalad worktree-add runs /tmp/worktrees/runs   # DataLad extension, same arguments
+python -m datalad_worktree add runs /tmp/worktrees/runs
+```
+
+Commands run from the superdataset root, or take `-d <path>` to name it.
+
+## `worktree add`
+
+```
+worktree add <branch> <worktree-path> [options]
+
+  <branch>                  branch to check out in every worktree
+  <worktree-path>           path for the superdataset worktree
+
+  -n, --dry-run             show what would be done, refusals included
+  -f, --force               replace an existing worktree even if it holds
+                            commits the main checkout lacks, discarding them
+  --follow-parent [<commit>]
+                            check each subdataset out at the commit its parent
+                            records; with a commit or tag, the superdataset too
+  --no-bindpaths            don't configure container bind mounts
+  --no-mtimes               don't copy mtimes (Snakemake markers included)
+                            from the main checkout
+  -d, --dataset <path>      superdataset root (default: current directory)
+```
+
+Creates a worktree for the superdataset and every installed subdataset, all on `branch`.
+
+```bash
+worktree add runs /tmp/worktrees/runs      # replaces the worktree if one is already there
+worktree add -f runs /tmp/worktrees/runs   # ... even if it holds commits never fetched
+```
+
+### Existing branches & worktrees
+
+`branch` always starts at each dataset's HEAD. It is created where it is missing, and reset where it exists. A worktree already at `<worktree-path>` is deleted, branch included, and created afresh. Uncommitted changes in it are discarded. 
+
+#### Pre-flight
+
+Before changing anything, `add` checks every dataset. If any check fails, it refuses and touches nothing, not even an existing worktree. It refuses when:
+
+- an existing worktree or branch holds commits the main checkout lacks, unless `-f`;
+- a directory at `<worktree-path>` is not one git knows as a worktree;
+- `branch` is checked out in a worktree at another path;
+- with `--follow-parent`, the commit can't be resolved, or a subdataset lacks the commit recorded for it.
+
+Only once every check has passed is an existing worktree deleted and the new ones created. If git still fails partway, for a reason no check can foresee (a stale lock, a full disk), `add` stops and deletes the worktrees it created. A replaced worktree stays deleted: it held nothing the main checkout lacks, or `-f` said to discard it.
+
+#### Dry run
+
+`-n` runs the pre-flight and stops, changing nothing. It reports the refusals, or what the real run would do: the worktree it would replace, and for each dataset where its worktree would go and whether `branch` would be new or reset.
+
+### `--follow-parent`
+
+By default each dataset checks out `branch` on its own, so a subdataset lands on its own branch tip. That may not be the commit the superdataset records for it. `--follow-parent` checks each subdataset out at the commit its parent records instead. All subdatasets and only those recorded by the target commit are created in the worktrees.
+
+```bash
+worktree add runs /tmp/wt --follow-parent           # as the superdataset records it now
+worktree add rerun /tmp/wt --follow-parent v1.0     # as commit or tag v1.0 recorded it
+```
+
+### Containers and mtimes
+
+The last two steps run over all the worktrees at once:
+
+- **Container bind mounts** are configured for containers registered in the superdataset. Containers registered in a subdataset are not configured. `--no-bindpaths` skips this step.
+- **mtimes are copied from the main checkout,** Snakemake's `.snakemake_timestamp` markers included. `--no-mtimes` skips this step.
+
+## `worktree fetch`
+
+```
+worktree fetch [target] [options]
+
+  [target]                  worktree path or branch name to fetch from
+                            (default: the checkout this worktree came from)
+
+  -n, --dry-run             show what would be merged, refusals included
+  --no-mtimes               don't copy mtimes (Snakemake markers included)
+                            from the target
+  -d, --dataset <path>      checkout to fetch into (default: current directory)
+```
+
+Brings `target`'s commits into the checkout you are standing in, then copies mtimes from `target`.
+
+```bash
+cd /data/my-project
+worktree fetch runs        # results home
+
+cd /tmp/worktrees/runs
+worktree fetch             # new code and inputs in
+```
+
+- No `target` from a main checkout is an error.
+- How commits come in:
+  - Where your side has no commits of its own, the fetch fast-forwards.
+  - Where both sides have commits, it merges (git >= 2.38; older git refuses).
+  - Where the merge would conflict, it refuses and names the paths. The usual case is both sides having moved the same subdataset.
+- Uncommitted changes block it only where the fetch would overwrite them, and those paths are named.
+- A dataset the worktree never changed is skipped.
+- A detached HEAD in any of the worktree's datasets refuses the whole fetch.
+
+## `worktree delete`
+
+```
+worktree delete <target> [options]
+
+  <target>                  worktree path or branch name
+
+  -n, --dry-run             show what would be deleted, refusals included
+  --keep-branch             keep the branch, and with it any commits the
+                            main checkout lacks
+  -f, --force               delete despite commits the main checkout
+                            lacks, discarding them
+  -d, --dataset <path>      superdataset root (default: current directory)
+```
+
+Deletes the worktree in every dataset, deepest first. It does not ask for confirmation, so use `-n` to preview.
+
+- The branch is deleted too, unless `--keep-branch`. Uncommitted changes are discarded, as `add` discards them.
+- Before deleting anything, `delete` checks every worktree. Unless `-f`, it refuses and deletes nothing if any worktree has commits the main checkout lacks. With `--keep-branch` these are kept on the branch, so only a worktree with a detached HEAD is refused for them.
+- `-n` runs the same checks and stops, so it never promises a deletion the real run refuses.
+- The main working tree is never deleted.
+
+## `worktree list`
+
+```
+worktree list [options]
+
+  -d, --dataset <path>      superdataset root (default: current directory)
+```
+
+`worktree` with no subcommand runs `list`, in the standalone CLI only. The listing shows datasets that have worktrees beyond the main checkout, grouped by the hierarchy each belongs to, with the main checkout's group first:
+
+```
+master
+  .           /mnt/Data/et_psychedelics/processed/2p
+runs
+  .           /mnt/Data/worktrees/2p-runs
+  code        /mnt/Data/worktrees/2p-runs/code (detached)
+  inputs/raw  /mnt/Data/worktrees/2p-runs/inputs/raw
+```
+
+A worktree directory deleted by other means (`rm -rf`) is pruned first, so it doesn't appear.

datalad_worktree-0.4.0/docs/design.md

@@ -0,0 +1,33 @@
+# Design notes
+
+## The premise: worktrees are ephemeral
+
+A worktree exists for one run: create it, run the pipeline, fetch the results home, throw it away. Every other decision follows from this.
+
+- **The main checkout is the only lasting place.** Results come home through `fetch`, and a new worktree starts from the main checkout's state, mtimes included both ways. Otherwise Snakemake reruns, on one side, work the other side already did.
+- **Creating is re-creating.** `add` replaces an existing worktree and starts every branch at HEAD. Going back to a particular state is possible with `--follow-parent <branch>`.
+- **Only unfetched commits are protected.** Uncommitted changes are discarded by `add` and `delete` alike. `-f` means "throw the commits away".
+- **All-or-nothing.** `add` and `delete` check every dataset before touching any, and `-n` runs those same checks and stops.
+
+## Containers
+
+- **Per-worktree config.** `add` writes the bind-mount option and a `cmdexec` carrying `{{bindpaths}}` with `git config --worktree`, so machine-specific paths stay out of the main checkout and out of history.
+- **One empty `bindpaths` is committed** to `.datalad/config`. `datalad run` records `{{bindpaths}}` unexpanded, so a record made in a worktree only reruns elsewhere if the name resolves there too, to nothing.
+- **Only the superdataset is configured** (#27). A container registered in a subdataset gets no bind paths.
+
+## mtimes
+
+Snakemake decides what to rerun by mtime, but git carries content, not timestamps:
+
+- **On `add`,** the checkout stamps every file "now", superdataset first, so every `code/` file ends up newer than the outputs derived from it.
+- **On `fetch`,** a merge moves only what it rewrites. That misses the grandparent directory a `directory()` output is judged by.
+- **On `datalad run` with identical outputs,** the local mtimes are correct, but since no content is shipped home upon `fetch`, the job is stale in the main checkout.
+
+So mtimes are copied from the other working tree: main → worktree on `add`, worktree → main on `fetch`. It covers directories, for `directory()` outputs, and Snakemake's gitignored `.snakemake_timestamp` markers (#39), which Snakemake reads instead of the directory and `snakemake --touch` stamps alone.
+
+What keeps it safe:
+
+- **Match on blob OID, not path,** so only byte-identical content inherits a timestamp. Skip paths dirty on either side.
+- **Never follow symlinks.** Annexed files point into an object store shared with the main repository.
+- **Skip unlocked annexed files.** Restamping one forces a re-hash through git-annex's clean filter: stamping 10099 symlinks cost the next `git status` 0.11 s, stamping 28 unlocked files (3.71 GB) cost it **152 s**.
+- **No mtimes from commit dates** (the `git-restore-mtime` approach). jj squash and rebase rewrite these repositories routinely, so every file would look new.