memseg 0.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. memseg-0.2.0/.gitignore +40 -0
  2. memseg-0.2.0/LICENSE +21 -0
  3. memseg-0.2.0/PKG-INFO +362 -0
  4. memseg-0.2.0/README.md +329 -0
  5. memseg-0.2.0/architecture.md +271 -0
  6. memseg-0.2.0/memseg/__init__.py +12 -0
  7. memseg-0.2.0/memseg/__main__.py +5 -0
  8. memseg-0.2.0/memseg/application/__init__.py +7 -0
  9. memseg-0.2.0/memseg/application/compare.py +164 -0
  10. memseg-0.2.0/memseg/application/semantic.py +101 -0
  11. memseg-0.2.0/memseg/application/use_cases.py +108 -0
  12. memseg-0.2.0/memseg/cli.py +518 -0
  13. memseg-0.2.0/memseg/domain/__init__.py +91 -0
  14. memseg-0.2.0/memseg/domain/metrics.py +145 -0
  15. memseg-0.2.0/memseg/domain/models.py +389 -0
  16. memseg-0.2.0/memseg/domain/ports.py +197 -0
  17. memseg-0.2.0/memseg/io/__init__.py +42 -0
  18. memseg-0.2.0/memseg/io/images.py +52 -0
  19. memseg-0.2.0/memseg/io/micrographs.py +199 -0
  20. memseg-0.2.0/memseg/io/previews.py +56 -0
  21. memseg-0.2.0/memseg/io/reports.py +269 -0
  22. memseg-0.2.0/memseg/io/side_by_side.py +86 -0
  23. memseg-0.2.0/memseg/io/sinks.py +71 -0
  24. memseg-0.2.0/memseg/io/sources.py +111 -0
  25. memseg-0.2.0/memseg/py.typed +0 -0
  26. memseg-0.2.0/memseg/registry.py +53 -0
  27. memseg-0.2.0/memseg/segmenters/__init__.py +43 -0
  28. memseg-0.2.0/memseg/segmenters/base.py +49 -0
  29. memseg-0.2.0/memseg/segmenters/components.py +50 -0
  30. memseg-0.2.0/memseg/semantic/__init__.py +38 -0
  31. memseg-0.2.0/memseg/semantic/base.py +27 -0
  32. memseg-0.2.0/memseg/semantic/threshold.py +92 -0
  33. memseg-0.2.0/pyproject.toml +104 -0
  34. memseg-0.2.0/tests/test_core.py +214 -0
  35. memseg-0.2.0/tests/test_plugins.py +61 -0
  36. memseg-0.2.0/tests/test_semantic.py +268 -0
@@ -0,0 +1,40 @@
1
+ # Python
2
+ __pycache__/
3
+ *.py[cod]
4
+ *.egg-info/
5
+ build/
6
+ dist/
7
+
8
+ # Environments
9
+ .venv/
10
+ .env
11
+
12
+ # Tools
13
+ .pytest_cache/
14
+ .mypy_cache/
15
+ .ruff_cache/
16
+ .ipynb_checkpoints/
17
+
18
+ # Editors / OS
19
+ .vscode/
20
+ .idea/
21
+ .DS_Store
22
+
23
+ # Model weights (large; share separately)
24
+ *.pt
25
+ *.pth
26
+ *.ckpt
27
+ *.sav
28
+
29
+ # Image data and run outputs (large)
30
+ *.tif
31
+ *.tiff
32
+ *.npz
33
+ out/
34
+ cmp/
35
+ eval_out/
36
+ prev/
37
+ compare_outs/
38
+
39
+ # Scripts
40
+ Scripts/
memseg-0.2.0/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Vaishak Karthikeyan
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
memseg-0.2.0/PKG-INFO ADDED
@@ -0,0 +1,362 @@
1
+ Metadata-Version: 2.5
2
+ Name: memseg
3
+ Version: 0.2.0
4
+ Summary: Framework for cryo-EM membrane segmentation: pluggable semantic and instance methods, scoring and a CLI
5
+ Project-URL: Homepage, https://github.com/vaishakk/memseg
6
+ Project-URL: Repository, https://github.com/vaishakk/memseg
7
+ Project-URL: Documentation, https://github.com/vaishakk/memseg/blob/main/architecture.md
8
+ Project-URL: Issues, https://github.com/vaishakk/memseg/issues
9
+ Author: Vaishak Karthikeyan
10
+ License-Expression: MIT
11
+ License-File: LICENSE
12
+ Keywords: cryo-em,electron-microscopy,image-analysis,instance-segmentation,membrane,segmentation
13
+ Classifier: Development Status :: 3 - Alpha
14
+ Classifier: Intended Audience :: Science/Research
15
+ Classifier: Operating System :: OS Independent
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Python :: 3 :: Only
18
+ Classifier: Programming Language :: Python :: 3.11
19
+ Classifier: Programming Language :: Python :: 3.12
20
+ Classifier: Programming Language :: Python :: 3.13
21
+ Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
22
+ Classifier: Topic :: Scientific/Engineering :: Image Processing
23
+ Classifier: Typing :: Typed
24
+ Requires-Python: >=3.11
25
+ Requires-Dist: numpy>=2.0
26
+ Requires-Dist: pillow>=10.0
27
+ Requires-Dist: scikit-image>=0.26.0
28
+ Requires-Dist: scipy>=1.13
29
+ Requires-Dist: tifffile>=2024.1.1
30
+ Provides-Extra: mrc
31
+ Requires-Dist: mrcfile>=1.5.4; extra == 'mrc'
32
+ Description-Content-Type: text/markdown
33
+
34
+ # memseg
35
+
36
+ A framework for **membrane segmentation** in cryo-EM, in two stages that can be used together or apart:
37
+
38
+ * **semantic**: micrograph → **binary membrane mask** (plus an optional probability map);
39
+ * **instance**: binary membrane mask → **instance label map**, with one label per membrane, or one
40
+ per layer of a multilamellar vesicle.
41
+
42
+ `memseg` provides everything around a segmentation method:
43
+
44
+ * reading micrographs (`.mrc` with header pixel size, `.tif`, ...), mask folders, labelled datasets and ground truth;
45
+ * writing label maps, colour previews and CSV/JSON reports;
46
+ * scoring:
47
+ * instances: precision, recall, F1 and IoU with Hungarian matching, plus pair AUROC;
48
+ * masks: pixel IoU, Dice, precision and recall;
49
+ * side-by-side comparison of methods, with timing;
50
+ * a command line interface.
51
+
52
+ **Methods are plugins.** Any installed package can add one through a Python entry point.
53
+ The CLI then offers it next to the built-in reference methods:
54
+
55
+ * `threshold` (semantic): a background-corrected threshold for dark lines;
56
+ * `components` (instance): every connected piece of membrane becomes one instance.
57
+
58
+ ## Installation
59
+
60
+ Requires Python ≥ 3.11.
61
+
62
+ ```bash
63
+ pip install memseg # or "memseg[mrc]" to read .mrc micrographs
64
+ # or, from a clone, with uv
65
+ uv sync --extra mrc
66
+ ```
67
+
68
+ This installs a `memseg` command; `python -m memseg` works the same way.
69
+
70
+ ## Usage
71
+
72
+ ```bash
73
+ memseg methods # list installed methods
74
+
75
+ # segment masks (files or folders, searched recursively) and write label maps
76
+ memseg segment masks/ -o out --method components --preview
77
+ memseg segment masks/ -o out --method components --set min_size=200
78
+
79
+ # score against ground truth: dataset folders containing masks/ and labels/
80
+ memseg evaluate data/val --method components --limit 20 -o eval_out
81
+
82
+ # run several methods on the same masks; compare speed, F1 and pair AUROC
83
+ memseg compare data/val -o cmp --methods components my_method --warmup
84
+ memseg compare masks/ --gt_dir labels/ -o cmp --methods components my_method --set components.min_size=200
85
+
86
+ # a method's own options
87
+ memseg segment --method components --help
88
+
89
+ # semantic: micrographs -> membrane masks, optionally -> instances, scored against GT masks
90
+ memseg semantic micrographs/ -o out --method threshold --preview
91
+ memseg semantic micrographs/ -o out --method threshold --instance components --gt_dir gt_masks/ --save_prob
92
+ memseg semantic --method threshold --instance components --help
93
+ ```
94
+
95
+ ### Common options
96
+
97
+ | option | meaning |
98
+ |---|---|
99
+ | `--method`, `-m` | method name (see `memseg methods`); default `components` |
100
+ | `--config FILE` / `--set KEY=VALUE` | method parameters from JSON / single overrides (repeatable) |
101
+ | `--preview_dir DIR` | write colour PNG previews |
102
+ | `--preview_overlay` | draw the method's overlay (skeleton / junctions) on instance previews; off by default |
103
+ | `--limit N`, `--sample N`, `--seed S` | process the first N / N random images |
104
+ | `--overwrite` (segment) | recompute images whose output already exists |
105
+ | `--save_raw` (segment) | also write method intermediates (`<name>_raw.npz`) |
106
+ | `--iou T` (evaluate, compare) | IoU threshold for a match (default 0.5) |
107
+
108
+ In `compare`, per-method parameters use `--set METHOD.KEY=VALUE` and `--config METHOD=FILE`.
109
+ `--methods` defaults to every installed method.
110
+
111
+ `semantic` has its own options:
112
+
113
+ | option | meaning |
114
+ |---|---|
115
+ | `--method`, `-m` | semantic method; default `threshold` |
116
+ | `--config` / `--set` | semantic method parameters |
117
+ | `--instance METHOD` | also run an instance method on each mask |
118
+ | `--instance_config` / `--instance_set` | instance method parameters |
119
+ | `--gt_dir DIR` | ground-truth masks with the same relative names: report pixel IoU / Dice |
120
+ | `--pixel_size A` | Å/px when the file has none (an MRC header value is used when set; 0 and 1.0 count as unset) |
121
+ | `--save_prob`, `--preview` | also write the probability map / PNG previews |
122
+ | `--preview_overlay` | draw the instance method's overlay on instance previews |
123
+
124
+ ### Inputs and outputs
125
+
126
+ * Inputs: `.tif .tiff .png .jpg .jpeg .bmp` masks (non-zero = membrane).
127
+ * Label maps: `<name>.tif` (uint16, uint32 if > 65 535 instances), mirroring the input folder structure.
128
+ * `summary.csv` per run (+ `summary.json` for `evaluate`, and for `semantic` with `--gt_dir`).
129
+ * `compare` writes:
130
+ * `side_by_side/<name>.png`: mask | GT | one panel per method;
131
+ * `<method>/<name>.tif`;
132
+ * `comparison.csv` and `comparison.json`.
133
+ * `semantic` writes:
134
+ * `<name>_mask.tif` (uint8, 255 = membrane), plus `<name>_prob.tif` and `<name>_mask.png` when asked;
135
+ * `instances/<name>.tif` (+ `_preview.png`) with `--instance`.
136
+
137
+ ## Writing your own method
138
+
139
+ A segmentation method is any class that implements `InstanceSegmenter.segment(mask) -> SegmentationResult`.
140
+ The framework handles everything else. A method must not do file I/O.
141
+
142
+ ### 1. Implement the method
143
+
144
+ The example below closes small gaps in the membrane before labelling its connected pieces:
145
+
146
+ ```python
147
+ # my_method/segmenter.py
148
+ import numpy as np
149
+ from scipy import ndimage
150
+
151
+ from memseg.domain import InstanceLabels, InstanceSegmenter, MembraneMask, SegmentationResult
152
+ from memseg.segmenters.base import SegmenterPlugin
153
+
154
+
155
+ class GapClosingSegmenter(InstanceSegmenter):
156
+ """Bridge gaps up to `gap` px, then one instance per connected piece."""
157
+
158
+ name = "gapclose"
159
+
160
+ def __init__(self, gap: int = 3):
161
+ self.gap = gap
162
+
163
+ def describe(self) -> str: # shown in logs
164
+ return f"gapclose (gap={self.gap})"
165
+
166
+ def segment(self, mask: MembraneMask) -> SegmentationResult: # mask.data: 2-D bool
167
+ closed = ndimage.binary_closing(mask.data, np.ones((2 * self.gap + 1,) * 2))
168
+ labels, n = ndimage.label(closed, structure=np.ones((3, 3)))
169
+ labels[~mask.data] = 0 # keep only original membrane pixels
170
+ return SegmentationResult(
171
+ labels=InstanceLabels.consecutive(labels), # relabel 1..N
172
+ method=self.name,
173
+ info=dict(pieces=n), # small scalars -> summary.csv column
174
+ )
175
+
176
+
177
+ class GapClosingPlugin(SegmenterPlugin):
178
+ """Connects the method to the CLI: its options and how to build it."""
179
+
180
+ name = "gapclose"
181
+ summary = "close small gaps, then connected components"
182
+
183
+ def add_arguments(self, group):
184
+ group.add_argument("--gap", type=int, default=3, help="largest gap to bridge (px)")
185
+
186
+ def build(self, args, overrides): # overrides = --config JSON + --set key=value
187
+ return GapClosingSegmenter(gap=overrides.get("gap", args.gap))
188
+ ```
189
+
190
+ `SegmentationResult` can also carry `artifacts` (large arrays, written by `--save_raw`) and an
191
+ `overlay` (skeleton / junctions drawn on previews with `--preview_overlay`).
192
+
193
+ ### 2. Register it with the CLI
194
+
195
+ **As an installable package (recommended).** Declare an entry point in your package's `pyproject.toml`:
196
+
197
+ ```toml
198
+ [project]
199
+ name = "my-method"
200
+ dependencies = ["memseg>=0.2"]
201
+
202
+ [project.entry-points."memseg.segmenters"]
203
+ gapclose = "my_method.segmenter:GapClosingPlugin"
204
+ ```
205
+
206
+ After `pip install -e .`, the method is available everywhere:
207
+
208
+ ```bash
209
+ memseg methods
210
+ memseg segment masks/ -o out --method gapclose --gap 5 --preview
211
+ memseg compare data/val -o cmp --methods components gapclose
212
+ ```
213
+
214
+ The plugin module is imported only when the method is used. A method that needs PyTorch therefore
215
+ does not slow down or break the other methods. Your package can be private: the framework only needs it installed.
216
+
217
+ **At run time, without packaging.**
218
+
219
+ ```python
220
+ import sys
221
+ from memseg import segmenters
222
+ from memseg.cli import main
223
+
224
+ segmenters.REGISTRY["gapclose"] = "my_method.segmenter:GapClosingPlugin"
225
+ sys.exit(main())
226
+ ```
227
+
228
+ ### 3. Or use it from Python with the built-in use cases and adapters
229
+
230
+ ```python
231
+ from memseg.application import EvaluateSegmenter, SegmentImages
232
+ from memseg.domain import CompositeSink
233
+ from memseg.io import (
234
+ ConsoleProgress,
235
+ CsvReportWriter,
236
+ FolderImageSource,
237
+ ImageFileGroundTruthReader,
238
+ ImageFileMaskReader,
239
+ LabelledDatasetSource,
240
+ LabelTiffSink,
241
+ PreviewSink,
242
+ )
243
+
244
+ from my_method.segmenter import GapClosingSegmenter
245
+
246
+ seg = GapClosingSegmenter(gap=5)
247
+
248
+ # segment: label maps + previews + summary.csv in out/
249
+ SegmentImages(
250
+ segmenter=seg,
251
+ source=FolderImageSource(["masks/"]), # files or folders
252
+ reader=ImageFileMaskReader(),
253
+ sink=CompositeSink([LabelTiffSink("out"), PreviewSink("out")]),
254
+ report=CsvReportWriter("out"),
255
+ progress=ConsoleProgress(),
256
+ ).run()
257
+
258
+ # evaluate against ground truth (dataset folders with masks/ and labels/)
259
+ outcomes, summary = EvaluateSegmenter(
260
+ segmenter=seg,
261
+ source=LabelledDatasetSource(["data/val"], limit=20),
262
+ reader=ImageFileMaskReader(),
263
+ gt_reader=ImageFileGroundTruthReader(),
264
+ progress=ConsoleProgress(),
265
+ ).run()
266
+ print(summary.f1, summary.mean_matched_iou)
267
+ ```
268
+
269
+ You can swap any adapter for your own implementation of the matching port in `memseg.domain`
270
+ (`ImageSource`, `MaskReader`, `ResultSink`, `ReportWriter`, `ProgressListener`). For example, a port
271
+ could read MRC files or write results into a CryoSPARC job.
272
+
273
+ ### Semantic methods
274
+
275
+ A semantic method implements `SemanticSegmenter.segment(image: Micrograph) -> SemanticResult`.
276
+
277
+ * `image.data` is a 2-D float32 array, and `image.pixel_size` is in Å/px, or `None` if unknown.
278
+ * The result holds:
279
+ * the `MembraneMask`;
280
+ * an optional `probability` map in [0, 1];
281
+ * `info` scalars, which go into `summary.csv`.
282
+ * Its plugin subclasses `memseg.semantic.base.SemanticSegmenterPlugin` and registers in its own
283
+ entry-point group.
284
+
285
+ Its CLI options share the command line with the instance method's, so prefix them, e.g. `--mynet_model`.
286
+
287
+ ```python
288
+ from memseg.domain import MembraneMask, SemanticResult, SemanticSegmenter
289
+ from memseg.semantic.base import SemanticSegmenterPlugin
290
+
291
+
292
+ class MyNet(SemanticSegmenter):
293
+ name = "mynet"
294
+
295
+ def segment(self, image):
296
+ prob = run_my_network(image.data, image.pixel_size) # your model
297
+ return SemanticResult(MembraneMask(prob > 0.5), self.name, probability=prob)
298
+
299
+
300
+ class MyNetPlugin(SemanticSegmenterPlugin):
301
+ name = "mynet"
302
+ summary = "my membrane network"
303
+
304
+ def build(self, args, overrides):
305
+ return MyNet()
306
+ ```
307
+
308
+ ```toml
309
+ [project.entry-points."memseg.semantic"]
310
+ mynet = "my_method.semantic:MyNetPlugin"
311
+ ```
312
+
313
+ From Python, `SegmentMicrographs` chains any semantic method with any instance method:
314
+
315
+ ```python
316
+ from memseg.application import SegmentMicrographs
317
+ from memseg.io import ConsoleMicrographProgress, FolderImageSource, MicrographFileReader, SemanticMaskSink
318
+ from memseg.segmenters.components import ConnectedComponentsSegmenter
319
+ from memseg.semantic.threshold import ThresholdSegmenter
320
+
321
+ outcomes, summary = SegmentMicrographs(
322
+ semantic=ThresholdSegmenter(),
323
+ instance=ConnectedComponentsSegmenter(min_size=2000), # optional
324
+ source=FolderImageSource(["micrographs/"], exts=[".mrc"]),
325
+ reader=MicrographFileReader(default_pixel_size=2.7),
326
+ semantic_sink=SemanticMaskSink("out", probability=True),
327
+ progress=ConsoleMicrographProgress(),
328
+ ).run()
329
+ ```
330
+
331
+ ## Project layout
332
+
333
+ ```
334
+ memseg/
335
+ ├── cli.py command line / composition root
336
+ ├── registry.py plugin discovery (built-ins + entry points), shared by both method families
337
+ ├── domain/ models, ports (interfaces), metrics — no I/O
338
+ ├── application/ use cases: SegmentImages, EvaluateSegmenter, CompareSegmenters, SegmentMicrographs
339
+ ├── io/ file readers/writers (incl. micrographs), previews, reports
340
+ ├── segmenters/ instance methods: registry, SegmenterPlugin, `components`
341
+ └── semantic/ semantic methods: registry, SemanticSegmenterPlugin, `threshold`
342
+ tests/ use-case tests with in-memory fakes, plugins, semantic pipeline, CLI
343
+ ```
344
+
345
+ See [architecture.md](https://github.com/vaishakk/memseg/blob/main/architecture.md) for the design, the pair-AUROC metric and how to add
346
+ methods or I/O adapters.
347
+
348
+ ## Development
349
+
350
+ ```bash
351
+ uv sync --extra mrc
352
+ uv run pytest # tests
353
+ uv run ruff check . # lint
354
+ uv run ruff format . # format
355
+ uv run mypy # type-check
356
+ ```
357
+
358
+ CI runs all four on every push and pull request.
359
+
360
+ ## License
361
+
362
+ MIT, see [LICENSE](https://github.com/vaishakk/memseg/blob/main/LICENSE).