scikit-ops 0.1.0.dev0__py3-none-any.whl → 0.1.0.dev1__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,48 @@
1
+ Metadata-Version: 2.5
2
+ Name: scikit-ops
3
+ Version: 0.1.0.dev1
4
+ Summary: Appose-powered ops, and the machinery for running them.
5
+ Project-URL: homepage, https://github.com/apposed/scikit-ops
6
+ Project-URL: documentation, https://github.com/apposed/scikit-ops/blob/main/README.md
7
+ Project-URL: source, https://github.com/apposed/scikit-ops
8
+ Project-URL: download, https://pypi.org/project/scikit-ops
9
+ Project-URL: tracker, https://github.com/apposed/appose/issues
10
+ Author: scikit-ops developers
11
+ License-Expression: BSD-2-Clause
12
+ Classifier: Development Status :: 2 - Pre-Alpha
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: Intended Audience :: Education
15
+ Classifier: Intended Audience :: Science/Research
16
+ Classifier: Operating System :: MacOS
17
+ Classifier: Operating System :: Microsoft :: Windows
18
+ Classifier: Operating System :: Unix
19
+ Classifier: Programming Language :: Python :: 3 :: Only
20
+ Classifier: Programming Language :: Python :: 3.10
21
+ Classifier: Programming Language :: Python :: 3.11
22
+ Classifier: Programming Language :: Python :: 3.12
23
+ Classifier: Programming Language :: Python :: 3.13
24
+ Classifier: Programming Language :: Python :: 3.14
25
+ Classifier: Topic :: Scientific/Engineering
26
+ Classifier: Topic :: Scientific/Engineering :: Image Processing
27
+ Classifier: Topic :: Scientific/Engineering :: Image Recognition
28
+ Classifier: Topic :: Software Development :: Libraries :: Python Modules
29
+ Classifier: Topic :: Utilities
30
+ Requires-Python: >=3.10
31
+ Requires-Dist: numpy
32
+ Description-Content-Type: text/markdown
33
+
34
+ # scikit-ops
35
+
36
+ Image-processing ops you can call directly, run in their own environment, or
37
+ use from napari and Fiji.
38
+
39
+ This is a development release. It does not work from PyPI yet, because the
40
+ packages it depends on are still in development. Early adopters can install
41
+ it from GitHub:
42
+
43
+ ```sh
44
+ pip install git+https://github.com/apposed/scikit-ops
45
+ ```
46
+
47
+ - Source: https://github.com/apposed/scikit-ops
48
+ - Docs: https://scikit-ops.readthedocs.io
@@ -60,7 +60,7 @@ skop/envs/segment-everything/pixi.toml,sha256=KMq2GUazmzN_Sw3dd5FuuAKMdiMCKtJTcX
60
60
  skop/envs/skimage/pixi.toml,sha256=X7W1E79E4UBHYSFiybeCgAYXvKumGCBT00reQnfNOHo,1305
61
61
  skop/envs/stardist-tf/pixi.toml,sha256=WyiniXNs6w2_-vz2XX1l_jr62vIVIG2mUOiLeyXV9dg,3656
62
62
  skop/envs/unseg-cv/pixi.toml,sha256=TwcYzpAnfBd5taAB0b-kRXBhbw5aXHoLzoKRaebvyDw,1065
63
- scikit_ops-0.1.0.dev0.dist-info/METADATA,sha256=-rNlEQzsKaScbKYxSG6vMSOVhhjFmy9z7D9FLx70ehQ,16089
64
- scikit_ops-0.1.0.dev0.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
65
- scikit_ops-0.1.0.dev0.dist-info/entry_points.txt,sha256=0gDhmxYQNjvBwJO4L0Q2fB-5fIEBbRVUS5u5heCYulQ,43
66
- scikit_ops-0.1.0.dev0.dist-info/RECORD,,
63
+ scikit_ops-0.1.0.dev1.dist-info/METADATA,sha256=OQS2Vkodps1aCA1rc3ZTRa6fZDL1lGzpMlv-NzsuNoA,1943
64
+ scikit_ops-0.1.0.dev1.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
65
+ scikit_ops-0.1.0.dev1.dist-info/entry_points.txt,sha256=0gDhmxYQNjvBwJO4L0Q2fB-5fIEBbRVUS5u5heCYulQ,43
66
+ scikit_ops-0.1.0.dev1.dist-info/RECORD,,
@@ -1,372 +0,0 @@
1
- Metadata-Version: 2.5
2
- Name: scikit-ops
3
- Version: 0.1.0.dev0
4
- Summary: Appose-powered ops, and the machinery for running them.
5
- Project-URL: homepage, https://github.com/apposed/scikit-ops
6
- Project-URL: documentation, https://github.com/apposed/scikit-ops/blob/main/README.md
7
- Project-URL: source, https://github.com/apposed/scikit-ops
8
- Project-URL: download, https://pypi.org/project/scikit-ops
9
- Project-URL: tracker, https://github.com/apposed/appose/issues
10
- Author: scikit-ops developers
11
- License-Expression: BSD-2-Clause
12
- Classifier: Development Status :: 2 - Pre-Alpha
13
- Classifier: Intended Audience :: Developers
14
- Classifier: Intended Audience :: Education
15
- Classifier: Intended Audience :: Science/Research
16
- Classifier: Operating System :: MacOS
17
- Classifier: Operating System :: Microsoft :: Windows
18
- Classifier: Operating System :: Unix
19
- Classifier: Programming Language :: Python :: 3 :: Only
20
- Classifier: Programming Language :: Python :: 3.10
21
- Classifier: Programming Language :: Python :: 3.11
22
- Classifier: Programming Language :: Python :: 3.12
23
- Classifier: Programming Language :: Python :: 3.13
24
- Classifier: Programming Language :: Python :: 3.14
25
- Classifier: Topic :: Scientific/Engineering
26
- Classifier: Topic :: Scientific/Engineering :: Image Processing
27
- Classifier: Topic :: Scientific/Engineering :: Image Recognition
28
- Classifier: Topic :: Software Development :: Libraries :: Python Modules
29
- Classifier: Topic :: Utilities
30
- Requires-Python: >=3.10
31
- Requires-Dist: numpy
32
- Description-Content-Type: text/markdown
33
-
34
- # scikit-ops
35
-
36
- A collection of image-processing **ops**, and `skop` — the machinery that runs
37
- them.
38
-
39
- An op is an ordinary Python function. The same function can be called directly,
40
- run in its own isolated environment through
41
- [Appose](https://github.com/apposed/appose), or wrapped in a GUI. Nothing about
42
- the op changes between those modes.
43
-
44
- Design notes — why the machinery is shaped this way, and what else was
45
- considered — are in [`docs/`](docs/README.md). How an op describes itself is
46
- [opspec](opspec/README.md), a separate package kept in this repository.
47
-
48
- ## Install
49
-
50
- Not on PyPI yet, so from GitHub:
51
-
52
- ```sh
53
- pip install "git+https://github.com/apposed/scikit-ops"
54
- ```
55
-
56
- That brings opspec with it. opspec on its own, standard library only, for
57
- a host or plugin that describes ops without running them:
58
-
59
- ```sh
60
- pip install "opspec @ git+https://github.com/apposed/scikit-ops#subdirectory=opspec"
61
- ```
62
-
63
- Working on the code instead, from a checkout: `uv sync`, which installs both,
64
- editable. See [Development](#development).
65
-
66
- ## Layout
67
-
68
- ```
69
- envs/<env-id>/pixi.toml environment definitions, named and shared between ops
70
- envs/<env-id>/init.py optional; runs in each worker before its I/O loop
71
- src/skop/ op-independent machinery
72
- src/skop/ops/<ns>.py an op namespace, when its ops share one environment
73
- src/skop/ops/<ns>/<op>.py an op namespace, when its ops do not
74
- opspec/ how an op describes itself: @op, OpSpec, roles, plans
75
- src/imgops/ napari/magicgui front ends (not published)
76
- ```
77
-
78
- Environments are keyed by ID, and several ops may declare the same one. Ops
79
- sharing an environment also share a warm worker process, unless one asks for
80
- `exclusive=True`.
81
-
82
- ## What's here
83
-
84
- | Op | Environment | Does |
85
- | --- | --- | --- |
86
- | `skop.ops.threshold:*` | `skimage` | Global thresholding: Otsu, isodata, Li, mean, minimum, triangle, Yen, multi-Otsu |
87
- | `skop.ops.smooth:*` | `skimage` | Smoothing and denoising: Gaussian, median, mean, Kuwahara, bilateral, TV, wavelet, NL-means, Butterworth |
88
- | `skop.ops.edges:*` | `skimage` | Derivative filters: Sobel, Scharr, Prewitt, Farid, Roberts, Laplace, DoG, unsharp mask, Frangi, Sato, Meijering |
89
- | `skop.ops.morphology:*` | `skimage` | Grayscale morphology: erosion, dilation, opening, closing, white/black tophat |
90
- | `skop.ops.kernels.psf:gaussian_psf` | `skimage` | Gaussian PSF, 2D or 3D |
91
- | `skop.ops.kernels.paraxial:paraxial_psf` | `skimage` | Paraxial PSF, from wavelength and NA |
92
- | `skop.ops.kernels.paraxial:paraxial_otf` | `skimage` | Paraxial OTF, the transform of the above |
93
- | `skop.ops.kernels.gibson_lanni:gibson_lanni` | `sdeconv` | Gibson-Lanni 3D PSF, with spherical aberration |
94
- | `skop.ops.deconvolve.richardson_lucy:richardson_lucy` | `skimage` | Richardson-Lucy, on the CPU |
95
- | `skop.ops.deconvolve.richardson_lucy_cupy:richardson_lucy_cupy` | `cupy` | Richardson-Lucy, on an NVIDIA GPU |
96
- | `skop.ops.segment.cellpose:cellpose` | `cellpose` | Cellpose segmentation |
97
- | `skop.ops.segment.stardist2d:stardist2d_fluo` | `stardist-tf` | StarDist 2D, pretrained fluorescence |
98
- | `skop.ops.segment.stardist2d:stardist2d_he` | `stardist-tf` | StarDist 2D, pretrained H&E |
99
- | `skop.ops.segment.stardist2d:stardist2d_custom` | `stardist-tf` | StarDist 2D, model loaded from a directory |
100
- | `skop.ops.train.stardist2d:train_stardist2d` | `stardist-tf` | StarDist 2D training, file pairs in, model path out |
101
- | `skop.ops.segment.starfun3d:segment_nuclei` | `stardist-tf` | StarDist 3D nuclei |
102
- | `skop.ops.labels:connect` | `skimage` | One label per object along an axis, by linking overlaps |
103
- | `skop.ops.generate:synthetic_nuclei` | `skimage` | Synthetic 3D test volume |
104
- | `skop.ops.segment.unseg:unseg` | `unseg-cv` | Unsupervised nuclei + cells |
105
- | `skop.ops.detect.fastsam:fastsam` | `pytorch` | Class-agnostic boxes, via FastSAM |
106
- | `skop.ops.detect.object_aware_yolo:object_aware_yolo` | `segment-everything` | Class-agnostic boxes, via MobileSAMv2 |
107
- | `skop.ops.detect.yolo:yolo` | `pytorch` | Tiled YOLO detection, with automatic GPU batching, confidences and classes |
108
- | `skop.ops.toy:*` | `minimal` | Exercises for skop itself |
109
-
110
- Those are the fully qualified op IDs, as `discover()` reports them. Callers
111
- import from the namespace instead, and never need to know whether it is one
112
- module or a package of them:
113
-
114
- ```python
115
- from skop.ops.threshold import otsu
116
- from skop.ops.segment import cellpose, stardist2d_fluo
117
- ```
118
-
119
- `stardist2d_fluo` and `segment_nuclei` share one TensorFlow build and one warm
120
- worker — the point of naming environments rather than tying them to ops.
121
- `unseg-cv` shares nothing with anything: it pins Python 3.9, numpy 1.24 and an
122
- old scikit-image, which is exactly why it needs an environment of its own.
123
-
124
- The two Richardson-Lucy ops are the same algorithm against different hardware,
125
- and there are two of them because `@op(env=...)` is fixed per function — a
126
- backend cannot be chosen at call time. Their signatures are identical and a
127
- test holds them numerically equivalent, so picking one is picking on speed.
128
-
129
- To list what the collection currently offers:
130
-
131
- ```python
132
- import oprunner
133
-
134
- specs, failures = oprunner.discover()
135
- ```
136
-
137
- For a large image and a YOLO `.pt` model:
138
-
139
- ```python
140
- from pathlib import Path
141
- from skop.ops.detect import yolo
142
-
143
- with oprunner.Runner() as runner:
144
- found = runner.run(
145
- yolo, image=image, weights=Path("best.pt"),
146
- object_size=100, overlap=0.15, gpu_fraction=0.9,
147
- )
148
- # found.boxes: [min_y, min_x, max_y, max_x]
149
- # found.confidences and found.classes: one value per box
150
- ```
151
-
152
- `object_size` is area in pixels²; `None` runs the whole image once. Areas
153
- outside 0.1%-50% of the image select square tiles with side
154
- `ceil(sqrt(1.5 * object_size / 0.001))`. Input size comes from the checkpoint.
155
- Tiles align with the image edges. Class-aware GreedyNMM matches SAHI 0.12.8:
156
- same-class boxes with IoS at least `merge_threshold` become an enclosing box
157
- with the maximum confidence. This includes matches within a tile. The default
158
- merge threshold is 0.5, independent of the per-tile NMS `iou` parameter.
159
- Outputs follow class order, then SAHI's keeper order within each class.
160
- A multiscale grid finds nearby candidates without an all-pairs matrix; the
161
- runtime needs no SAHI dependency. To run the optional reference comparisons:
162
-
163
- ```sh
164
- uv run --with sahi==0.12.8 pytest tests/test_yolo.py -k sahi
165
- ```
166
-
167
- GPU batches are calibrated after warm-up, with intermediate sizes before
168
- larger batches. Measured peaks adjust the size up or down; an out-of-memory
169
- retry establishes an upper bound rather than permanently halving throughput.
170
- CUDA measures the reserved allocator footprint; MPS samples driver memory.
171
- One CPU batch is prepared ahead of inference, using pinned buffers for CUDA.
172
- Inference transfers only detection data back to the host. The image must fit
173
- in host RAM; only the current resized batch enters the GPU.
174
-
175
- ## Writing an op
176
-
177
- ```python
178
- from typing import Annotated, NamedTuple
179
-
180
- import numpy as np
181
- from opspec.op import op
182
- from opspec.types import ImageOf, LabelsOf
183
-
184
-
185
- class Result(NamedTuple):
186
- labels: LabelsOf[np.ndarray]
187
- count: int
188
-
189
-
190
- @op(env="stardist-tf")
191
- def segment(
192
- image: ImageOf[np.ndarray],
193
- prob_thresh: Annotated[
194
- float, {"widget_type": "FloatSlider", "min": 0.0, "max": 1.0}
195
- ] = 0.5,
196
- ) -> Result:
197
- """Segment nuclei."""
198
- from stardist.models import StarDist3D # NB: heavy imports go in the body.
199
-
200
- ...
201
- ```
202
-
203
- Two rules, and both exist for the same reason — ops are discovered by importing
204
- them into a minimal environment that has little more than numpy:
205
-
206
- 1. **Heavy imports go inside the function body.** A module-scope
207
- `import tensorflow` makes the op undiscoverable.
208
- 2. **Type your inputs and outputs with plain types.** Annotations *are*
209
- evaluated at discovery time, so `-> tf.Tensor` fails even if the import is
210
- in the body.
211
-
212
- Multiple outputs come from a `NamedTuple`: its field names become the names of
213
- the op's outputs. `progress(...)` and `cancel_requested()`, from
214
- `opspec.runtime`, report to whoever is running the op, and do nothing when it
215
- is called directly.
216
-
217
- ### Saying what an array is
218
-
219
- Every op here passes `np.ndarray` around, which tells a front end nothing
220
- about how to display it. opspec's types add that missing half:
221
-
222
- | Write | Role | napari shows it as |
223
- | --- | --- | --- |
224
- | `ImageOf[np.ndarray]` | `Role.image` | an Image layer |
225
- | `LabelsOf[np.ndarray]` | `Role.labels` | a Labels layer |
226
- | `PointsOf[np.ndarray]` | `Role.points` | a Points layer |
227
- | `VectorsOf[np.ndarray]` | `Role.vectors` | a Vectors layer |
228
- | `TracksOf[np.ndarray]` | `Role.tracks` | a Tracks layer |
229
- | `BoxesOf[np.ndarray]` | `Role.boxes` | a Shapes layer |
230
- | `MasksOf[np.ndarray]` | `Role.masks` | a Labels layer, once projected |
231
-
232
- `MasksOf` is the odd one: an `(N, Y, X)` stack of masks that may overlap, so
233
- it is not a label image and no viewer shows it directly. A front end projects
234
- it with `ophelper.masks` — flattened to 2-D, or kept as a rotatable 3-D stack —
235
- and gets to offer that as a choice, since re-projecting costs a numpy call
236
- rather than another run of the op.
237
-
238
- They are `Annotated[np.ndarray, ...]`, so nothing else changes: the codec
239
- still sees an array, direct callers still get an array, and `ParamSpec.type`
240
- is still `np.ndarray`. They compose with the rest —
241
- `Out[LabelsOf[np.ndarray]]` and
242
- `Annotated[LabelsOf[np.ndarray], {"label": "Nuclei"}]` both work.
243
-
244
- Roles are advisory and never guessed. An unannotated array reports
245
- `role is None`, and deciding what to do with that is the front end's business,
246
- not opspec's. Nothing in `opspec.types` imports napari; the roles simply
247
- mirror `napari.types` so that mapping is a lookup rather than a judgement call,
248
- and `Role` is equally the hook a Fiji or plain-magicgui front end reads.
249
-
250
- ### Where it goes
251
-
252
- Ops live in a namespace under `skop/ops/`, named for what they do. How the
253
- namespace is laid out inside depends on its ops, and the split point is the
254
- environment:
255
-
256
- - **One module** when the ops are variations on one call sharing one
257
- environment — every `skop.ops.threshold` op is a few lines against
258
- `skimage.filters`, so they share `threshold.py`.
259
- - **A package, one module per op**, when they do not — `skop.ops.segment`
260
- spans three environments and each op carries its own models and types, so
261
- each gets a file and `segment/__init__.py` re-exports them.
262
-
263
- An op needing several files of its own becomes a package
264
- (`segment/unseg/__init__.py` plus siblings). Underscore-prefixed modules are
265
- private and never imported by discovery, which is where shared helpers and
266
- vendored code belong.
267
-
268
- Because `from skop.ops.<ns> import <op>` reads the same either way, a
269
- namespace can grow from a module into a package without breaking callers.
270
-
271
- ### Computation forms
272
-
273
- Following [SciJava Ops](https://ops.scijava.org/en/latest/Concepts.html), an op
274
- is one of three forms, declared through its signature:
275
-
276
- | Form | Signature | Meaning |
277
- | --- | --- | --- |
278
- | function | `def f(image) -> Result` | allocates and returns its output |
279
- | computer | `def f(image, out: Out[np.ndarray])` | fills a buffer the caller owns |
280
- | inplace | `def f(image: Mut[np.ndarray])` | mutates its input |
281
-
282
- `Out` parameters are hidden from generated GUIs — a user is never asked for an
283
- output buffer. Adapting between forms is not implemented yet: a computer or
284
- inplace op must be handed its buffers.
285
-
286
- ## Running an op
287
-
288
- ```python
289
- import oprunner
290
- from skop.ops import toy
291
-
292
- # Directly, in this process, using this process's dependencies.
293
- toy.add(2, 3)
294
-
295
- # Or in the op's own environment, in a worker process.
296
- with oprunner.Runner() as runner:
297
- runner.run(toy.add, a=2, b=3)
298
- ```
299
-
300
- The first run of an environment builds it, which takes a while. Later runs
301
- reuse it, since Appose keys environments by name.
302
-
303
- Because that build happens inside `run()`, a caller with a progress bar can
304
- subscribe to it:
305
-
306
- ```python
307
- runner.subscribe_build_progress(lambda title, current, maximum: ...)
308
- runner.subscribe_build_output(lambda text: ...)
309
- runner.subscribe_build_error(lambda text: ...)
310
- ```
311
-
312
- These pass straight through to Appose's builder. Progress titles name the
313
- phase — `Solving`, `Installing conda packages`, `Downloading PyPI packages`,
314
- `Installing PyPI packages`, `Done` — and come with a real denominator.
315
-
316
- Two things worth knowing. Subscribing to progress is what *enables* it:
317
- Appose's `PixiInstallMonitor` runs pixi under `-vv` to read its phase
318
- transitions, and only wires up when a progress subscriber exists. And
319
- `build_error` is the stderr *stream* rather than a failure report — pixi
320
- writes its ordinary status there, and under `-vv` its whole debug log as
321
- well. A build that actually fails raises out of `run()`.
322
-
323
- This needs Appose newer than 0.11, which is why `pyproject.toml` currently
324
- sources it from the main branch.
325
-
326
- ## Development
327
-
328
- ```sh
329
- uv sync --all-groups
330
- uv run pytest
331
- uv run ruff check --fix && uv run ruff format
332
- ```
333
-
334
- ### Testing the ops for real
335
-
336
- Ops are exercised end to end in their own environments, in `test_ops_e2e.py`.
337
- Each such test declares what it needs with `@pytest.mark.env("<env-id>")`, and
338
- is skipped when that environment is not built — so a plain `pytest` run stays
339
- fast and offline, testing against whatever happens to be installed already.
340
-
341
- To run them all, building whatever is missing:
342
-
343
- ```sh
344
- uv run pytest --build-envs # everything; slow the first time
345
- uv run pytest --build-envs -m env # only the op tests
346
- uv run pytest -m "not env" # only the fast ones
347
- ```
348
-
349
- Running the notebooks and the interactive tests is a separate setup, since they
350
- need a host environment and the ops do not run in it:
351
- [notebooks/README.md](notebooks/README.md).
352
-
353
- ### Checking that the environments still solve
354
-
355
- Building is slow; solving is not. `scripts/check_envs.py` resolves every
356
- `envs/*/pixi.toml` without downloading anything, and exits non-zero if one is
357
- unsatisfiable:
358
-
359
- ```sh
360
- python scripts/check_envs.py
361
- ```
362
-
363
- This catches what the tests structurally cannot: they run *inside* an
364
- environment, and so cannot tell you whether that environment could still be
365
- created today. One already built keeps working from its lock file however
366
- broken its `pixi.toml` becomes — which is how an unsolvable `envs/pytorch`
367
- survived a week. Run it after editing anything under `envs/`.
368
-
369
- CI runs `--build-envs`, since these ops are the whole point of the project and
370
- nothing else validates them against the stacks they target. The flag is also
371
- settable as `PYTEST_ADDOPTS=--build-envs`, which is handy when the pytest
372
- invocation is not yours to edit.