patchworks 2.8.0__tar.gz → 3.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (149) hide show
  1. patchworks-3.0.0/.github/min-versions.txt +6 -0
  2. patchworks-3.0.0/.github/workflows/lint.yml +69 -0
  3. patchworks-3.0.0/.github/workflows/test.yml +110 -0
  4. patchworks-3.0.0/.pre-commit-config.yaml +11 -0
  5. {patchworks-2.8.0 → patchworks-3.0.0}/PKG-INFO +34 -12
  6. {patchworks-2.8.0 → patchworks-3.0.0}/README.md +16 -3
  7. patchworks-3.0.0/benchmarks/bench.py +186 -0
  8. patchworks-3.0.0/benchmarks/compare.py +46 -0
  9. {patchworks-2.8.0 → patchworks-3.0.0}/docs/api/io.md +6 -0
  10. {patchworks-2.8.0 → patchworks-3.0.0}/docs/api/plugins/ome_zarr.md +1 -1
  11. patchworks-3.0.0/docs/api/provenance.md +17 -0
  12. patchworks-3.0.0/docs/api/seams.md +9 -0
  13. {patchworks-2.8.0 → patchworks-3.0.0}/docs/getting_started.md +1 -1
  14. patchworks-3.0.0/docs/guide/cli.md +40 -0
  15. {patchworks-2.8.0 → patchworks-3.0.0}/docs/guide/gpu_distributed.md +17 -0
  16. patchworks-3.0.0/docs/guide/launcher.md +252 -0
  17. {patchworks-2.8.0 → patchworks-3.0.0}/docs/guide/merging.md +81 -0
  18. {patchworks-2.8.0 → patchworks-3.0.0}/docs/guide/ome_zarr_napari.md +25 -4
  19. {patchworks-2.8.0 → patchworks-3.0.0}/docs/guide/performance.md +40 -1
  20. {patchworks-2.8.0 → patchworks-3.0.0}/docs/guide/pitfalls.md +3 -1
  21. {patchworks-2.8.0 → patchworks-3.0.0}/docs/guide/snakemake.md +38 -1
  22. {patchworks-2.8.0 → patchworks-3.0.0}/mkdocs.yml +4 -0
  23. {patchworks-2.8.0 → patchworks-3.0.0}/pyproject.toml +59 -9
  24. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/__init__.py +26 -8
  25. patchworks-3.0.0/src/patchworks/_autotune.py +168 -0
  26. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/_chunks.py +200 -35
  27. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/_cluster.py +15 -7
  28. patchworks-3.0.0/src/patchworks/_core.py +1229 -0
  29. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/_distributed.py +60 -0
  30. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/_gpu.py +20 -12
  31. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/_io.py +194 -44
  32. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/_merge.py +474 -117
  33. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/_occupancy.py +3 -2
  34. patchworks-3.0.0/src/patchworks/_postprocess.py +232 -0
  35. patchworks-3.0.0/src/patchworks/_provenance.py +86 -0
  36. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/_relabel.py +44 -25
  37. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/_relations.py +57 -36
  38. patchworks-3.0.0/src/patchworks/_seams.py +186 -0
  39. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/_volume_filter.py +30 -36
  40. patchworks-3.0.0/src/patchworks/cli.py +407 -0
  41. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/plugins/cellpose.py +15 -15
  42. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/plugins/dog.py +209 -19
  43. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/plugins/napari.py +45 -8
  44. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/plugins/ome_zarr.py +469 -86
  45. patchworks-3.0.0/src/patchworks/py.typed +0 -0
  46. {patchworks-2.8.0 → patchworks-3.0.0}/tests/test_allocation.py +135 -0
  47. patchworks-3.0.0/tests/test_autotune.py +45 -0
  48. patchworks-3.0.0/tests/test_cli.py +67 -0
  49. patchworks-3.0.0/tests/test_core.py +1032 -0
  50. {patchworks-2.8.0 → patchworks-3.0.0}/tests/test_distributed.py +110 -0
  51. {patchworks-2.8.0 → patchworks-3.0.0}/tests/test_dog.py +119 -15
  52. {patchworks-2.8.0 → patchworks-3.0.0}/tests/test_gpu.py +36 -0
  53. patchworks-3.0.0/tests/test_launcher.py +274 -0
  54. patchworks-3.0.0/tests/test_napari.py +372 -0
  55. {patchworks-2.8.0 → patchworks-3.0.0}/tests/test_ome_zarr.py +161 -46
  56. patchworks-3.0.0/tests/test_postprocess.py +99 -0
  57. {patchworks-2.8.0 → patchworks-3.0.0}/tests/test_pw.py +54 -0
  58. {patchworks-2.8.0 → patchworks-3.0.0}/tests/test_relations.py +28 -0
  59. patchworks-3.0.0/tests/test_remote.py +79 -0
  60. {patchworks-2.8.0 → patchworks-3.0.0}/tests/test_run_multi.py +92 -3
  61. patchworks-3.0.0/tests/test_seams.py +59 -0
  62. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/config/common.yaml +6 -0
  63. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/config/config.yaml +14 -0
  64. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/config/config_cilia.yaml +10 -0
  65. patchworks-3.0.0/workflow/launcher/README.md +83 -0
  66. patchworks-3.0.0/workflow/launcher/app.py +660 -0
  67. patchworks-3.0.0/workflow/launcher/clusters.yaml +32 -0
  68. patchworks-3.0.0/workflow/launcher/launcher_core.py +548 -0
  69. patchworks-3.0.0/workflow/launcher/requirements.txt +3 -0
  70. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/pixi.toml +34 -4
  71. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/scripts/_pw.py +81 -1
  72. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/scripts/convert.py +3 -1
  73. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/scripts/merge.py +69 -2
  74. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/scripts/prepare_tiles.py +17 -2
  75. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/scripts/segment_tile.py +34 -2
  76. patchworks-3.0.0/workflow/viewer/pixi.toml +36 -0
  77. patchworks-2.8.0/.github/workflows/lint.yml +0 -35
  78. patchworks-2.8.0/src/patchworks/_core.py +0 -647
  79. patchworks-2.8.0/src/patchworks/_postprocess.py +0 -101
  80. patchworks-2.8.0/tests/test_core.py +0 -428
  81. patchworks-2.8.0/tests/test_napari.py +0 -181
  82. patchworks-2.8.0/tests/test_postprocess.py +0 -41
  83. {patchworks-2.8.0 → patchworks-3.0.0}/.github/workflows/docs.yml +0 -0
  84. {patchworks-2.8.0 → patchworks-3.0.0}/.github/workflows/release.yml +0 -0
  85. {patchworks-2.8.0 → patchworks-3.0.0}/.gitignore +0 -0
  86. {patchworks-2.8.0 → patchworks-3.0.0}/.markdownlint-cli2.yaml +0 -0
  87. {patchworks-2.8.0 → patchworks-3.0.0}/LICENSE +0 -0
  88. {patchworks-2.8.0 → patchworks-3.0.0}/cliff.toml +0 -0
  89. {patchworks-2.8.0 → patchworks-3.0.0}/docs/api/chunks.md +0 -0
  90. {patchworks-2.8.0 → patchworks-3.0.0}/docs/api/cluster.md +0 -0
  91. {patchworks-2.8.0 → patchworks-3.0.0}/docs/api/merge_tile_labels.md +0 -0
  92. {patchworks-2.8.0 → patchworks-3.0.0}/docs/api/plugins/cellpose.md +0 -0
  93. {patchworks-2.8.0 → patchworks-3.0.0}/docs/api/plugins/dog.md +0 -0
  94. {patchworks-2.8.0 → patchworks-3.0.0}/docs/api/plugins/napari.md +0 -0
  95. {patchworks-2.8.0 → patchworks-3.0.0}/docs/api/postprocess.md +0 -0
  96. {patchworks-2.8.0 → patchworks-3.0.0}/docs/api/relabel.md +0 -0
  97. {patchworks-2.8.0 → patchworks-3.0.0}/docs/api/tile_process.md +0 -0
  98. {patchworks-2.8.0 → patchworks-3.0.0}/docs/api/volume_filter.md +0 -0
  99. {patchworks-2.8.0 → patchworks-3.0.0}/docs/assets/logo.png +0 -0
  100. {patchworks-2.8.0 → patchworks-3.0.0}/docs/examples/cellpose_2d.md +0 -0
  101. {patchworks-2.8.0 → patchworks-3.0.0}/docs/examples/cellpose_2d.py +0 -0
  102. {patchworks-2.8.0 → patchworks-3.0.0}/docs/examples/cellpose_3d.md +0 -0
  103. {patchworks-2.8.0 → patchworks-3.0.0}/docs/examples/cellpose_3d.py +0 -0
  104. {patchworks-2.8.0 → patchworks-3.0.0}/docs/examples/custom.md +0 -0
  105. {patchworks-2.8.0 → patchworks-3.0.0}/docs/examples/custom_method.py +0 -0
  106. {patchworks-2.8.0 → patchworks-3.0.0}/docs/examples/dog.md +0 -0
  107. {patchworks-2.8.0 → patchworks-3.0.0}/docs/examples/dog.py +0 -0
  108. {patchworks-2.8.0 → patchworks-3.0.0}/docs/examples/standalone_merge.md +0 -0
  109. {patchworks-2.8.0 → patchworks-3.0.0}/docs/examples/stardist.md +0 -0
  110. {patchworks-2.8.0 → patchworks-3.0.0}/docs/examples/stardist_2d.py +0 -0
  111. {patchworks-2.8.0 → patchworks-3.0.0}/docs/guide/custom_segmentation.md +0 -0
  112. {patchworks-2.8.0 → patchworks-3.0.0}/docs/guide/label_relations.md +0 -0
  113. {patchworks-2.8.0 → patchworks-3.0.0}/docs/guide/measurements.md +0 -0
  114. {patchworks-2.8.0 → patchworks-3.0.0}/docs/guide/skip_empty.md +0 -0
  115. {patchworks-2.8.0 → patchworks-3.0.0}/docs/guide/tiling.md +0 -0
  116. {patchworks-2.8.0 → patchworks-3.0.0}/docs/index.md +0 -0
  117. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/_notify.py +0 -0
  118. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/_progress.py +0 -0
  119. {patchworks-2.8.0 → patchworks-3.0.0}/src/patchworks/plugins/__init__.py +0 -0
  120. {patchworks-2.8.0 → patchworks-3.0.0}/tests/ngff_schemas/0.4/image.schema +0 -0
  121. {patchworks-2.8.0 → patchworks-3.0.0}/tests/ngff_schemas/0.4/label.schema +0 -0
  122. {patchworks-2.8.0 → patchworks-3.0.0}/tests/ngff_schemas/0.4/ome.schema +0 -0
  123. {patchworks-2.8.0 → patchworks-3.0.0}/tests/ngff_schemas/0.5/_version.schema +0 -0
  124. {patchworks-2.8.0 → patchworks-3.0.0}/tests/ngff_schemas/0.5/image.schema +0 -0
  125. {patchworks-2.8.0 → patchworks-3.0.0}/tests/ngff_schemas/0.5/label.schema +0 -0
  126. {patchworks-2.8.0 → patchworks-3.0.0}/tests/ngff_schemas/0.5/ome.schema +0 -0
  127. {patchworks-2.8.0 → patchworks-3.0.0}/tests/ngff_schemas/README.md +0 -0
  128. {patchworks-2.8.0 → patchworks-3.0.0}/tests/test_cellpose.py +0 -0
  129. {patchworks-2.8.0 → patchworks-3.0.0}/tests/test_notify.py +0 -0
  130. {patchworks-2.8.0 → patchworks-3.0.0}/tests/test_occupancy.py +0 -0
  131. {patchworks-2.8.0 → patchworks-3.0.0}/tests/test_progress.py +0 -0
  132. {patchworks-2.8.0 → patchworks-3.0.0}/tests/test_volume_filter.py +0 -0
  133. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/README.md +0 -0
  134. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/Snakefile +0 -0
  135. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/config/config_cyto.yaml +0 -0
  136. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/config/config_nuclei.yaml +0 -0
  137. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/config/multi.yaml +0 -0
  138. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/profile/slurm/config.yaml +0 -0
  139. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/rules/common.smk +0 -0
  140. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/rules/convert.smk +0 -0
  141. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/rules/merge.smk +0 -0
  142. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/rules/segment.smk +0 -0
  143. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/scripts/build_occupancy.py +0 -0
  144. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/scripts/export_iso.py +0 -0
  145. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/scripts/fetch_model.py +0 -0
  146. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/scripts/relate.py +0 -0
  147. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/scripts/reshard_store.py +0 -0
  148. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/scripts/run_multi.py +0 -0
  149. {patchworks-2.8.0 → patchworks-3.0.0}/workflow/scripts/view.py +0 -0
@@ -0,0 +1,6 @@
1
+ # Oldest releases patchworks claims to support (pyproject floors).
2
+ # The "minimum versions" CI job installs against these.
3
+ dask==2024.10.0
4
+ numpy==1.26.0
5
+ zarr==3.1.3
6
+ scipy==1.11.1
@@ -0,0 +1,69 @@
1
+ name: Lint
2
+
3
+ on:
4
+ push:
5
+ branches:
6
+ - main
7
+ pull_request:
8
+
9
+ jobs:
10
+ ruff:
11
+ runs-on: ubuntu-latest
12
+ steps:
13
+ - uses: actions/checkout@v4
14
+
15
+ - name: ruff check
16
+ uses: astral-sh/ruff-action@v3
17
+ with:
18
+ version: "0.15.18"
19
+ args: "check"
20
+
21
+ - name: ruff format --check
22
+ uses: astral-sh/ruff-action@v3
23
+ with:
24
+ version: "0.15.18"
25
+ args: "format --check"
26
+
27
+ markdownlint:
28
+ runs-on: ubuntu-latest
29
+ steps:
30
+ - uses: actions/checkout@v4
31
+
32
+ - name: markdownlint
33
+ uses: DavidAnson/markdownlint-cli2-action@v20
34
+ with:
35
+ globs: "**/*.md"
36
+
37
+ docs:
38
+ # The deploy workflow only runs on main; catch a broken reference or a
39
+ # docstring mkdocstrings cannot render before it gets there.
40
+ runs-on: ubuntu-latest
41
+ steps:
42
+ - uses: actions/checkout@v4
43
+
44
+ - uses: actions/setup-python@v5
45
+ with:
46
+ python-version: "3.12"
47
+ cache: pip
48
+
49
+ - name: Install docs dependencies
50
+ run: pip install "mkdocs-material>=9.0" "mkdocstrings[python]>=0.24" .
51
+
52
+ - name: mkdocs build --strict
53
+ run: mkdocs build --strict
54
+
55
+ mypy:
56
+ runs-on: ubuntu-latest
57
+ steps:
58
+ - uses: actions/checkout@v4
59
+
60
+ - uses: actions/setup-python@v5
61
+ with:
62
+ python-version: "3.12"
63
+ cache: pip
64
+
65
+ - name: Install
66
+ run: pip install -e ".[dev]"
67
+
68
+ - name: mypy (config in pyproject.toml)
69
+ run: mypy
@@ -0,0 +1,110 @@
1
+ name: Tests
2
+
3
+ on:
4
+ push:
5
+ branches:
6
+ - main
7
+ pull_request:
8
+
9
+ jobs:
10
+ pytest:
11
+ runs-on: ubuntu-latest
12
+ strategy:
13
+ fail-fast: false
14
+ matrix:
15
+ # The floor (requires-python) and the newest release.
16
+ python-version: ["3.11", "3.12", "3.13", "3.14"]
17
+ steps:
18
+ - uses: actions/checkout@v4
19
+
20
+ - uses: actions/setup-python@v5
21
+ with:
22
+ python-version: ${{ matrix.python-version }}
23
+ cache: pip
24
+
25
+ - name: Install
26
+ run: pip install -e ".[dev]"
27
+
28
+ - name: pytest
29
+ run: pytest -q
30
+
31
+ - name: docstring examples
32
+ run: pytest -q --doctest-modules src
33
+
34
+ other-platforms:
35
+ # Not yet verified on macOS or Windows (no CI ran there before), so this
36
+ # job reports without failing the build; promote it into the matrix
37
+ # above once it is green.
38
+ runs-on: ${{ matrix.os }}
39
+ continue-on-error: true
40
+ strategy:
41
+ fail-fast: false
42
+ matrix:
43
+ os: [macos-latest, windows-latest]
44
+ steps:
45
+ - uses: actions/checkout@v4
46
+
47
+ - uses: actions/setup-python@v5
48
+ with:
49
+ python-version: "3.12"
50
+ cache: pip
51
+
52
+ - name: Install
53
+ run: pip install -e ".[dev]"
54
+
55
+ - name: pytest
56
+ run: pytest -q
57
+
58
+ minimum-versions:
59
+ # The pyproject floors are only true if something runs on them.
60
+ runs-on: ubuntu-latest
61
+ steps:
62
+ - uses: actions/checkout@v4
63
+
64
+ - uses: actions/setup-python@v5
65
+ with:
66
+ python-version: "3.11"
67
+ cache: pip
68
+
69
+ - name: Install at the declared minimums
70
+ run: pip install -e ".[dev]" -c .github/min-versions.txt
71
+
72
+ - name: pytest
73
+ run: pytest -q
74
+
75
+ benchmarks:
76
+ # Head vs base on the same runner, reported in the job summary with a
77
+ # warning per case >25% slower. Never fails the build: shared runners
78
+ # are too noisy for a hard gate.
79
+ if: github.event_name == 'pull_request'
80
+ runs-on: ubuntu-latest
81
+ continue-on-error: true
82
+ steps:
83
+ - uses: actions/checkout@v4
84
+ with:
85
+ fetch-depth: 0
86
+
87
+ - uses: actions/setup-python@v5
88
+ with:
89
+ python-version: "3.12"
90
+ cache: pip
91
+
92
+ - name: Benchmark the pull request
93
+ run: |
94
+ cp -r benchmarks "$RUNNER_TEMP/bench"
95
+ pip install -e .
96
+ python "$RUNNER_TEMP/bench/bench.py" "$RUNNER_TEMP/head.json"
97
+
98
+ - name: Benchmark the base branch
99
+ run: |
100
+ git worktree add "$RUNNER_TEMP/base" "origin/${{ github.base_ref }}"
101
+ python -m venv "$RUNNER_TEMP/venv-base"
102
+ "$RUNNER_TEMP/venv-base/bin/pip" install "$RUNNER_TEMP/base"
103
+ "$RUNNER_TEMP/venv-base/bin/python" "$RUNNER_TEMP/bench/bench.py" \
104
+ "$RUNNER_TEMP/base.json"
105
+
106
+ - name: Compare
107
+ run: |
108
+ python "$RUNNER_TEMP/bench/compare.py" \
109
+ "$RUNNER_TEMP/base.json" "$RUNNER_TEMP/head.json" \
110
+ | tee -a "$GITHUB_STEP_SUMMARY"
@@ -0,0 +1,11 @@
1
+ # Same checks as .github/workflows/lint.yml, at the same versions.
2
+ repos:
3
+ - repo: https://github.com/astral-sh/ruff-pre-commit
4
+ rev: v0.15.18
5
+ hooks:
6
+ - id: ruff-check
7
+ - id: ruff-format
8
+ - repo: https://github.com/DavidAnson/markdownlint-cli2
9
+ rev: v0.18.1
10
+ hooks:
11
+ - id: markdownlint-cli2
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.5
2
2
  Name: patchworks
3
- Version: 2.8.0
3
+ Version: 3.0.0
4
4
  Summary: Tiled processing of arbitrarily large images with globally consistent labels
5
5
  Project-URL: Homepage, https://github.com/imcf/patchworks
6
6
  Project-URL: Issues, https://github.com/imcf/patchworks/issues
@@ -8,20 +8,21 @@ Author-email: Laurent Guerard <laurent.guerard@unibas.ch>
8
8
  License: GPL-3.0-only
9
9
  License-File: LICENSE
10
10
  Keywords: bioimage,chunked,dask,image processing,segmentation,tiling,zarr
11
- Classifier: Development Status :: 5 - Production/Stable
11
+ Classifier: Development Status :: 4 - Beta
12
12
  Classifier: Intended Audience :: Science/Research
13
13
  Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
14
14
  Classifier: Programming Language :: Python :: 3
15
- Classifier: Programming Language :: Python :: 3.9
16
- Classifier: Programming Language :: Python :: 3.10
17
15
  Classifier: Programming Language :: Python :: 3.11
18
16
  Classifier: Programming Language :: Python :: 3.12
17
+ Classifier: Programming Language :: Python :: 3.13
18
+ Classifier: Programming Language :: Python :: 3.14
19
19
  Classifier: Topic :: Scientific/Engineering :: Image Processing
20
- Requires-Python: >=3.9
21
- Requires-Dist: dask[array]>=2023.1.0
22
- Requires-Dist: numpy>=1.24
23
- Requires-Dist: scipy>=1.9
24
- Requires-Dist: zarr>=2.14
20
+ Requires-Python: >=3.11
21
+ Requires-Dist: dask[array]>=2024.10.0
22
+ Requires-Dist: numpy>=1.26
23
+ Requires-Dist: psutil
24
+ Requires-Dist: scipy>=1.11
25
+ Requires-Dist: zarr>=3.1.3
25
26
  Provides-Extra: all
26
27
  Requires-Dist: bioio; extra == 'all'
27
28
  Requires-Dist: bioio-bioformats; extra == 'all'
@@ -30,6 +31,7 @@ Requires-Dist: bioio-lif; extra == 'all'
30
31
  Requires-Dist: bioio-nd2; extra == 'all'
31
32
  Requires-Dist: bioio-ome-tiff; extra == 'all'
32
33
  Requires-Dist: bioio-tifffile; extra == 'all'
34
+ Requires-Dist: dask[diagnostics,distributed]; extra == 'all'
33
35
  Requires-Dist: glasbey; extra == 'all'
34
36
  Requires-Dist: imaris-ims-file-reader; extra == 'all'
35
37
  Requires-Dist: ipykernel<7; extra == 'all'
@@ -58,11 +60,16 @@ Provides-Extra: cellpose4
58
60
  Requires-Dist: cellpose>=4; extra == 'cellpose4'
59
61
  Provides-Extra: dev
60
62
  Requires-Dist: jsonschema>=4.18; extra == 'dev'
63
+ Requires-Dist: mypy>=1.10; extra == 'dev'
64
+ Requires-Dist: openpyxl; extra == 'dev'
61
65
  Requires-Dist: psutil; extra == 'dev'
62
66
  Requires-Dist: pytest; extra == 'dev'
63
67
  Requires-Dist: pytest-cov; extra == 'dev'
68
+ Requires-Dist: ruff==0.15.18; extra == 'dev'
64
69
  Requires-Dist: scikit-image; extra == 'dev'
65
70
  Requires-Dist: tqdm; extra == 'dev'
71
+ Provides-Extra: distributed
72
+ Requires-Dist: dask[diagnostics,distributed]; extra == 'distributed'
66
73
  Provides-Extra: docs
67
74
  Requires-Dist: mkdocs-material>=9.0; extra == 'docs'
68
75
  Requires-Dist: mkdocstrings[python]>=0.24; extra == 'docs'
@@ -83,6 +90,8 @@ Requires-Dist: napari-chunked-regionprops>=0.1.0; extra == 'napari'
83
90
  Requires-Dist: napari[all]>=0.7.1; extra == 'napari'
84
91
  Requires-Dist: numpy<2.5; extra == 'napari'
85
92
  Requires-Dist: pyqt6<6.10; extra == 'napari'
93
+ Provides-Extra: remote
94
+ Requires-Dist: fsspec[gcs,http,s3]; extra == 'remote'
86
95
  Provides-Extra: workflow
87
96
  Requires-Dist: openpyxl; extra == 'workflow'
88
97
  Requires-Dist: snakemake-executor-plugin-slurm; extra == 'workflow'
@@ -135,6 +144,7 @@ Optional extras:
135
144
 
136
145
  ```bash
137
146
  pip install "patchworks[gpu]" # GPU VRAM querying (nvidia-ml-py)
147
+ pip install "patchworks[distributed]" # Dask cluster + live dashboard
138
148
  pip install "patchworks[cellpose]" # Cellpose plugin (>=3.0, v3 or v4)
139
149
  pip install "patchworks[cellpose3]" # Cellpose plugin, pinned to v3.x
140
150
  pip install "patchworks[cellpose4]" # Cellpose plugin, pinned to v4+
@@ -276,6 +286,18 @@ from disk, so terabyte volumes convert in bounded RAM. See the
276
286
 
277
287
  ---
278
288
 
289
+ ## From the command line
290
+
291
+ ```bash
292
+ patchworks convert scan.czi scan.zarr
293
+ patchworks segment scan.zarr --method cellpose --model cyto3 --diameter 30 --gpu
294
+ patchworks info scan.zarr
295
+ patchworks view scan.zarr
296
+ ```
297
+
298
+ See [Command line](https://imcf.one/patchworks/guide/cli/) for every method
299
+ and flag.
300
+
279
301
  ## Common patterns
280
302
 
281
303
  ### Auto-size tiles from available memory
@@ -409,12 +431,12 @@ Full docs, guides and tutorials: **<https://imcf.one/patchworks/>**
409
431
 
410
432
  ## Requirements
411
433
 
412
- - Python ≥ 3.9
413
- - dask[array], numpy, zarr, scipy
434
+ - Python ≥ 3.11
435
+ - dask[array], numpy, zarr ≥ 3.1.3, scipy, psutil (RAM-aware tile and worker
436
+ sizing)
414
437
 
415
438
  Optional:
416
439
 
417
- - `psutil` — accurate RAM sizing for `tile_shape="auto"`
418
440
  - `nvidia-ml-py` — accurate GPU VRAM sizing
419
441
  - `tqdm` — progress bars
420
442
  - `cellpose` — Cellpose plugin, v3 or v4 (`patchworks[cellpose]`);
@@ -44,6 +44,7 @@ Optional extras:
44
44
 
45
45
  ```bash
46
46
  pip install "patchworks[gpu]" # GPU VRAM querying (nvidia-ml-py)
47
+ pip install "patchworks[distributed]" # Dask cluster + live dashboard
47
48
  pip install "patchworks[cellpose]" # Cellpose plugin (>=3.0, v3 or v4)
48
49
  pip install "patchworks[cellpose3]" # Cellpose plugin, pinned to v3.x
49
50
  pip install "patchworks[cellpose4]" # Cellpose plugin, pinned to v4+
@@ -185,6 +186,18 @@ from disk, so terabyte volumes convert in bounded RAM. See the
185
186
 
186
187
  ---
187
188
 
189
+ ## From the command line
190
+
191
+ ```bash
192
+ patchworks convert scan.czi scan.zarr
193
+ patchworks segment scan.zarr --method cellpose --model cyto3 --diameter 30 --gpu
194
+ patchworks info scan.zarr
195
+ patchworks view scan.zarr
196
+ ```
197
+
198
+ See [Command line](https://imcf.one/patchworks/guide/cli/) for every method
199
+ and flag.
200
+
188
201
  ## Common patterns
189
202
 
190
203
  ### Auto-size tiles from available memory
@@ -318,12 +331,12 @@ Full docs, guides and tutorials: **<https://imcf.one/patchworks/>**
318
331
 
319
332
  ## Requirements
320
333
 
321
- - Python ≥ 3.9
322
- - dask[array], numpy, zarr, scipy
334
+ - Python ≥ 3.11
335
+ - dask[array], numpy, zarr ≥ 3.1.3, scipy, psutil (RAM-aware tile and worker
336
+ sizing)
323
337
 
324
338
  Optional:
325
339
 
326
- - `psutil` — accurate RAM sizing for `tile_shape="auto"`
327
340
  - `nvidia-ml-py` — accurate GPU VRAM sizing
328
341
  - `tqdm` — progress bars
329
342
  - `cellpose` — Cellpose plugin, v3 or v4 (`patchworks[cellpose]`);
@@ -0,0 +1,186 @@
1
+ """Timing benchmarks for patchworks' hot paths.
2
+
3
+ python benchmarks/bench.py out.json # run, write JSON
4
+ python benchmarks/compare.py base.json head.json
5
+
6
+ Each case runs a few times on synthetic but realistic data and keeps the
7
+ fastest run (the least disturbed by a noisy machine). A case the installed
8
+ patchworks cannot run (e.g. a feature newer than a base branch) is recorded
9
+ as null, so a head/base comparison still works across such changes.
10
+ """
11
+
12
+ from __future__ import annotations
13
+
14
+ import json
15
+ import shutil
16
+ import sys
17
+ import tempfile
18
+ import time
19
+ from pathlib import Path
20
+
21
+ import numpy as np
22
+ import zarr
23
+
24
+ REPEATS = 3
25
+
26
+
27
+ def _blob_labels(shape, n, seed=0, r=4):
28
+ """Many small blobs, labelled per tile as a stage store holds them."""
29
+ from scipy import ndimage as ndi
30
+
31
+ rng = np.random.default_rng(seed)
32
+ mask = np.zeros(shape, bool)
33
+ radii = np.array([max(1, min(r, (d - 1) // 2)) for d in shape])
34
+ centres = rng.integers(radii, np.array(shape) - radii, (n, len(shape)))
35
+ for c in centres:
36
+ mask[tuple(slice(x - q, x + q) for x, q in zip(c, radii))] = True
37
+ return ndi.label(mask)[0].astype("int32")
38
+
39
+
40
+ def _staged(tmp, shape=(8, 1024, 1024), tile=(8, 256, 256), n=4000):
41
+ """A stage store: dense 1..n ids per tile, plus each tile's count."""
42
+ lab = _blob_labels(shape, n)
43
+ path = str(tmp / "stage.zarr")
44
+ arr = zarr.open_group(path, mode="w").create_array(
45
+ "staged", shape=shape, chunks=tile, dtype="int32"
46
+ )
47
+ import itertools
48
+
49
+ # Only public API and plain numpy here: the base branch of a comparison
50
+ # may predate any internal helper.
51
+ counts = []
52
+ grid = [range(0, n, t) for n, t in zip(shape, tile)]
53
+ for starts in itertools.product(*grid):
54
+ sl = tuple(slice(o, o + t) for o, t in zip(starts, tile))
55
+ ids = np.unique(lab[sl])
56
+ lut = np.zeros(int(ids.max()) + 1, dtype="int32")
57
+ lut[ids[ids > 0]] = np.arange(1, int((ids > 0).sum()) + 1)
58
+ block = lut[lab[sl]]
59
+ arr[sl] = block
60
+ counts.append(int(block.max()))
61
+ return path, counts
62
+
63
+
64
+ def case_merge_touch(tmp):
65
+ from patchworks._merge import zarr_native_merge
66
+
67
+ path, counts = _staged(tmp)
68
+ t = time.perf_counter()
69
+ zarr_native_merge(
70
+ path,
71
+ "staged",
72
+ str(tmp / "out.zarr"),
73
+ "labels",
74
+ n_workers=2,
75
+ label_counts=counts,
76
+ )
77
+ return time.perf_counter() - t
78
+
79
+
80
+ def case_merge_renumber(tmp):
81
+ """The merge without per-tile counts: includes the renumber pass."""
82
+ from patchworks._merge import zarr_native_merge
83
+
84
+ path, _ = _staged(tmp)
85
+ t = time.perf_counter()
86
+ zarr_native_merge(
87
+ path, "staged", str(tmp / "out.zarr"), "labels", n_workers=2
88
+ )
89
+ return time.perf_counter() - t
90
+
91
+
92
+ def case_image_pyramid(tmp):
93
+ from patchworks.plugins.ome_zarr import to_ome_zarr
94
+
95
+ img = np.random.default_rng(0).integers(0, 4000, (16, 1024, 1024))
96
+ img = img.astype("uint16")
97
+ t = time.perf_counter()
98
+ to_ome_zarr(img, tmp / "img.zarr", axes="zyx", n_levels=5, progress=False)
99
+ return time.perf_counter() - t
100
+
101
+
102
+ def case_label_pyramid(tmp):
103
+ from patchworks.plugins.ome_zarr import (
104
+ add_pyramid,
105
+ to_ome_zarr,
106
+ write_labels,
107
+ )
108
+
109
+ lab = _blob_labels((16, 1024, 1024), 6000)
110
+ store = to_ome_zarr(
111
+ np.zeros((1, 8, 8), "uint16"),
112
+ tmp / "i.zarr",
113
+ axes="zyx",
114
+ n_levels=1,
115
+ progress=False,
116
+ )
117
+ write_labels(store, lab, name="l", n_levels=1, progress=False)
118
+ t = time.perf_counter()
119
+ add_pyramid(f"{store}/labels/l", n_levels=5, progress=False)
120
+ return time.perf_counter() - t
121
+
122
+
123
+ def case_filter_by_size(tmp):
124
+ from patchworks import filter_labels_by_size
125
+
126
+ lab = _blob_labels((16, 1024, 1024), 6000)
127
+ path = str(tmp / "f.zarr")
128
+ z = zarr.open_group(path, mode="w").create_array(
129
+ "l", shape=lab.shape, chunks=(16, 256, 256), dtype="int32"
130
+ )
131
+ z[:] = lab
132
+ t = time.perf_counter()
133
+ filter_labels_by_size(path, "l", min_voxels=50)
134
+ return time.perf_counter() - t
135
+
136
+
137
+ def case_tile_process(tmp):
138
+ import dask.array as da
139
+ from scipy import ndimage as ndi
140
+
141
+ from patchworks import tile_process
142
+
143
+ img = (_blob_labels((8, 512, 512), 1500) > 0).astype("uint16") * 1000
144
+ arr = da.from_array(img, chunks=(8, 128, 128))
145
+ t = time.perf_counter()
146
+ tile_process(
147
+ arr,
148
+ lambda b: ndi.label(b > 500)[0].astype("int32"),
149
+ overlap=8,
150
+ write_to=tmp / "o.zarr",
151
+ progress=False,
152
+ )
153
+ return time.perf_counter() - t
154
+
155
+
156
+ CASES = {
157
+ name[5:]: fn
158
+ for name, fn in sorted(globals().items())
159
+ if name.startswith("case_")
160
+ }
161
+
162
+
163
+ def run() -> dict[str, float | None]:
164
+ results: dict[str, float | None] = {}
165
+ for name, fn in CASES.items():
166
+ times = []
167
+ for _ in range(REPEATS):
168
+ tmp = Path(tempfile.mkdtemp(prefix="pwbench_"))
169
+ try:
170
+ times.append(fn(tmp))
171
+ except Exception as exc: # a case this version cannot run
172
+ print(f"{name}: skipped ({type(exc).__name__}: {exc})")
173
+ times = []
174
+ break
175
+ finally:
176
+ shutil.rmtree(tmp, ignore_errors=True)
177
+ results[name] = min(times) if times else None
178
+ if times:
179
+ print(f"{name}: {results[name]:.3f}s")
180
+ return results
181
+
182
+
183
+ if __name__ == "__main__":
184
+ out = run()
185
+ if len(sys.argv) > 1:
186
+ Path(sys.argv[1]).write_text(json.dumps(out, indent=2))
@@ -0,0 +1,46 @@
1
+ """Compare two benchmark runs as a Markdown table.
2
+
3
+ python benchmarks/compare.py base.json head.json [--threshold 1.25]
4
+
5
+ Prints the table (append it to $GITHUB_STEP_SUMMARY in CI) and a GitHub
6
+ warning annotation per case that got slower than *threshold*. It never fails:
7
+ shared CI runners are too noisy for a hard gate, so this is for a human to
8
+ read on every pull request.
9
+ """
10
+
11
+ from __future__ import annotations
12
+
13
+ import argparse
14
+ import json
15
+ from pathlib import Path
16
+
17
+
18
+ def main() -> int:
19
+ ap = argparse.ArgumentParser()
20
+ ap.add_argument("base")
21
+ ap.add_argument("head")
22
+ ap.add_argument("--threshold", type=float, default=1.25)
23
+ args = ap.parse_args()
24
+ base = json.loads(Path(args.base).read_text())
25
+ head = json.loads(Path(args.head).read_text())
26
+
27
+ print("| case | base (s) | head (s) | head / base |")
28
+ print("| --- | ---: | ---: | ---: |")
29
+ slow = []
30
+ for name in sorted(set(base) | set(head)):
31
+ b, h = base.get(name), head.get(name)
32
+ ratio = h / b if b and h else None
33
+ mark = ""
34
+ if ratio is not None and ratio > args.threshold:
35
+ mark = " ⚠️"
36
+ slow.append((name, ratio))
37
+ fmt = lambda v: "n/a" if v is None else f"{v:.3f}" # noqa: E731
38
+ r = "n/a" if ratio is None else f"{ratio:.2f}x{mark}"
39
+ print(f"| {name} | {fmt(b)} | {fmt(h)} | {r} |")
40
+ for name, ratio in slow:
41
+ print(f"::warning title=benchmark::{name} is {ratio:.2f}x slower")
42
+ return 0
43
+
44
+
45
+ if __name__ == "__main__":
46
+ raise SystemExit(main())
@@ -17,3 +17,9 @@ used as a skip list. See [Skipping empty tiles](../guide/skip_empty.md).
17
17
  ::: patchworks.tile_occupancy
18
18
 
19
19
  ::: patchworks.auto_empty_threshold
20
+
21
+ ## Compression
22
+
23
+ ::: patchworks.compression
24
+
25
+ ::: patchworks.set_compression
@@ -42,7 +42,7 @@ matches the store, and reads both.
42
42
  ::: patchworks.plugins.ome_zarr.ngff_version
43
43
 
44
44
  Every writer takes an `ngff_version=` keyword: `"auto"` (default) follows the
45
- installed zarr — 0.5 on v3, 0.4 on v2 — and `"0.4"` pins the older, zarr-v2
45
+ installed zarr (0.5, since patchworks requires zarr-python 3) and `"0.4"` pins the older, zarr-v2
46
46
  layout. Writing into an existing store always follows *that store's* format,
47
47
  so a label pyramid added later can never disagree with the image it sits in.
48
48
  NGFF 0.6 is released but not written: RFC-5 replaces `axes` with
@@ -0,0 +1,17 @@
1
+ # Provenance
2
+
3
+ Every label image patchworks writes records how it was made: the
4
+ segmentation function and its bound settings, tiling, overlap, stitching,
5
+ level, channel, codec, the input, the library versions and a UTC timestamp.
6
+ It sits in the label group's attrs under `"patchworks"` (for `write_to=`
7
+ stores, on the labels array), so it travels with the data.
8
+
9
+ ```python
10
+ from patchworks import read_provenance
11
+
12
+ read_provenance("scan.zarr/labels/cilia_labels")["settings"]["custom"]
13
+ ```
14
+
15
+ ::: patchworks.read_provenance
16
+
17
+ ::: patchworks.provenance
@@ -0,0 +1,9 @@
1
+ # Seam report
2
+
3
+ ::: patchworks.seam_report
4
+
5
+ ## Choosing the overlap
6
+
7
+ ::: patchworks.suggest_overlap
8
+
9
+ ::: patchworks.object_f1
@@ -2,7 +2,7 @@
2
2
 
3
3
  ## Installation
4
4
 
5
- patchworks can be installed from PyPI on all operating systems, for Python ≥ 3.9.
5
+ patchworks can be installed from PyPI on all operating systems, for Python ≥ 3.11.
6
6
 
7
7
  !!! tip "Virtual environment (recommended)"
8
8
  We recommend creating a dedicated environment:
@@ -0,0 +1,40 @@
1
+ # Command line
2
+
3
+ Installing patchworks also installs a `patchworks` command: the same building
4
+ blocks as the Python API, for a one-off run without writing a script or a
5
+ Snakemake config.
6
+
7
+ ```bash
8
+ # Convert anything bioio reads (or an existing store) to a pyramidal OME-ZARR
9
+ patchworks convert scan.czi scan.zarr
10
+
11
+ # Segment it; labels go into scan.zarr/labels/labels
12
+ patchworks segment scan.zarr --method cellpose --model cyto3 --diameter 30 --gpu
13
+
14
+ # What is in the store: levels, chunks, codecs, calibration, label images
15
+ patchworks info scan.zarr
16
+
17
+ # Did the tiling leave marks in the labels?
18
+ patchworks seams scan.zarr/labels/labels --tile-shape 16,1024,1024
19
+
20
+ # Look at the result (needs patchworks[napari])
21
+ patchworks view scan.zarr
22
+ ```
23
+
24
+ ## Segmentation methods
25
+
26
+ | `--method` | What it does | Main flags |
27
+ | --- | --- | --- |
28
+ | `threshold` (default) | one global threshold (Otsu over the image unless `--threshold`), then connected components | `--threshold` |
29
+ | `cellpose` | [Cellpose](../api/plugins/cellpose.md) | `--model`, `--diameter`, `--do-3d`, `--gpu` |
30
+ | `dog` | [difference of Gaussians](../api/plugins/dog.md) | `--low-sigma`, `--high-sigma`, `--threshold` |
31
+ | `custom` | any importable function | `--fn module:function`, `--fn-kwargs '{"k": 1}'` |
32
+
33
+ Cellpose's anisotropy and the DoG plugin's voxel size are read from the
34
+ store's own calibration, at the level being segmented.
35
+
36
+ Tiling and stitching take the same options as
37
+ [`tile_process`](../api/tile_process.md): `--tile-shape` (`z,y,x`, `auto` or
38
+ `none`), `--overlap` (`N` or `z,y,x`), `--stitch iou`, `--resume`,
39
+ `--skip-empty`, `--level`, `--channel` (an index, or `none` for every channel),
40
+ `--compression`. Run `patchworks <command> --help` for the full list.