skb-arrow 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- skb_arrow-0.1.0/.gitignore +7 -0
- skb_arrow-0.1.0/LICENSE +32 -0
- skb_arrow-0.1.0/PKG-INFO +109 -0
- skb_arrow-0.1.0/README.md +63 -0
- skb_arrow-0.1.0/docs/DESIGN.md +500 -0
- skb_arrow-0.1.0/docs/architecture.md +55 -0
- skb_arrow-0.1.0/docs/capabilities.md +254 -0
- skb_arrow-0.1.0/docs/errors.md +79 -0
- skb_arrow-0.1.0/docs/protocol.md +89 -0
- skb_arrow-0.1.0/docs/transport.md +70 -0
- skb_arrow-0.1.0/pyproject.toml +92 -0
- skb_arrow-0.1.0/src/skb_arrow/__init__.py +0 -0
- skb_arrow-0.1.0/src/skb_arrow/capabilities/__init__.py +0 -0
- skb_arrow-0.1.0/src/skb_arrow/capabilities/_tables.py +351 -0
- skb_arrow-0.1.0/src/skb_arrow/capabilities/ancombc.py +159 -0
- skb_arrow-0.1.0/src/skb_arrow/capabilities/echo.py +8 -0
- skb_arrow-0.1.0/src/skb_arrow/capabilities/mantel.py +38 -0
- skb_arrow-0.1.0/src/skb_arrow/cli.py +32 -0
- skb_arrow-0.1.0/src/skb_arrow/doctor.py +191 -0
- skb_arrow-0.1.0/src/skb_arrow/errors.py +146 -0
- skb_arrow-0.1.0/src/skb_arrow/host.py +270 -0
- skb_arrow-0.1.0/src/skb_arrow/protocol.py +175 -0
- skb_arrow-0.1.0/src/skb_arrow/registry.py +176 -0
- skb_arrow-0.1.0/src/skb_arrow/transport.py +161 -0
- skb_arrow-0.1.0/tests/client.py +171 -0
- skb_arrow-0.1.0/tests/data/atlas/LICENSE.txt +27 -0
- skb_arrow-0.1.0/tests/data/atlas/README.md +63 -0
- skb_arrow-0.1.0/tests/data/atlas/atlas_ancombc_global.tsv +22 -0
- skb_arrow-0.1.0/tests/data/atlas/atlas_ancombc_main.tsv +148 -0
- skb_arrow-0.1.0/tests/data/atlas/atlas_feature_table.csv +301 -0
- skb_arrow-0.1.0/tests/data/atlas/atlas_meta_data.csv +301 -0
- skb_arrow-0.1.0/tests/data/interfaces.json +140 -0
- skb_arrow-0.1.0/tests/fixture_host.py +227 -0
- skb_arrow-0.1.0/tests/segment_timing.py +128 -0
- skb_arrow-0.1.0/tests/test_ancombc.py +431 -0
- skb_arrow-0.1.0/tests/test_capabilities.py +196 -0
- skb_arrow-0.1.0/tests/test_cli.py +111 -0
- skb_arrow-0.1.0/tests/test_doctor.py +315 -0
- skb_arrow-0.1.0/tests/test_errors.py +282 -0
- skb_arrow-0.1.0/tests/test_host.py +655 -0
- skb_arrow-0.1.0/tests/test_large.py +98 -0
- skb_arrow-0.1.0/tests/test_mantel.py +261 -0
- skb_arrow-0.1.0/tests/test_protocol.py +480 -0
- skb_arrow-0.1.0/tests/test_registry.py +209 -0
- skb_arrow-0.1.0/tests/test_soak.py +132 -0
- skb_arrow-0.1.0/tests/test_tables.py +713 -0
- skb_arrow-0.1.0/tests/test_transport.py +456 -0
- skb_arrow-0.1.0/uv.lock +1034 -0
skb_arrow-0.1.0/LICENSE
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
=================================
|
|
2
|
+
The skb-arrow licensing terms
|
|
3
|
+
=================================
|
|
4
|
+
|
|
5
|
+
The skb-arrow project is licensed under the terms of the Modified BSD License
|
|
6
|
+
(also known as New or Revised BSD), as follows:
|
|
7
|
+
|
|
8
|
+
Copyright (c) 2026-, The skb-arrow Development Team <damcdonald@ucsd.edu>
|
|
9
|
+
|
|
10
|
+
All rights reserved.
|
|
11
|
+
|
|
12
|
+
Redistribution and use in source and binary forms, with or without
|
|
13
|
+
modification, are permitted provided that the following conditions are met:
|
|
14
|
+
* Redistributions of source code must retain the above copyright
|
|
15
|
+
notice, this list of conditions and the following disclaimer.
|
|
16
|
+
* Redistributions in binary form must reproduce the above copyright
|
|
17
|
+
notice, this list of conditions and the following disclaimer in the
|
|
18
|
+
documentation and/or other materials provided with the distribution.
|
|
19
|
+
* Neither the name of the skb-arrow development team nor the names of its
|
|
20
|
+
contributors may be used to endorse or promote products derived from this
|
|
21
|
+
software without specific prior written permission.
|
|
22
|
+
|
|
23
|
+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
|
|
24
|
+
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
|
|
25
|
+
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
|
26
|
+
DISCLAIMED. IN NO EVENT SHALL THE SKB-ARROW DEVELOPMENT TEAM BE LIABLE FOR
|
|
27
|
+
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
|
|
28
|
+
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
|
|
29
|
+
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
|
|
30
|
+
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
|
31
|
+
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
|
|
32
|
+
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
skb_arrow-0.1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: skb-arrow
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Process host exposing scikit-bio to non-Python callers over Arrow
|
|
5
|
+
Project-URL: Repository, https://github.com/the-miint/skb-arrow
|
|
6
|
+
Project-URL: Documentation, https://github.com/the-miint/skb-arrow/tree/main/docs
|
|
7
|
+
Project-URL: Issues, https://github.com/the-miint/skb-arrow/issues
|
|
8
|
+
Author-email: Daniel McDonald <damcdonald@ucsd.edu>
|
|
9
|
+
License-Expression: BSD-3-Clause
|
|
10
|
+
License-File: LICENSE
|
|
11
|
+
Keywords: apache-arrow,bioinformatics,microbiome,scikit-bio
|
|
12
|
+
Classifier: Development Status :: 3 - Alpha
|
|
13
|
+
Classifier: Operating System :: MacOS
|
|
14
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
16
|
+
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
|
|
17
|
+
Requires-Python: ==3.14.*
|
|
18
|
+
Requires-Dist: array-api-compat==1.15.0
|
|
19
|
+
Requires-Dist: biom-format==2.1.18
|
|
20
|
+
Requires-Dist: certifi==2026.7.22
|
|
21
|
+
Requires-Dist: charset-normalizer==3.5.2
|
|
22
|
+
Requires-Dist: click==8.5.0
|
|
23
|
+
Requires-Dist: decorator==5.3.1
|
|
24
|
+
Requires-Dist: formulaic==1.2.2
|
|
25
|
+
Requires-Dist: h5py==3.16.0
|
|
26
|
+
Requires-Dist: idna==3.20
|
|
27
|
+
Requires-Dist: interface-meta==2.0.1
|
|
28
|
+
Requires-Dist: llvmlite==0.50.0
|
|
29
|
+
Requires-Dist: narwhals==2.26.0
|
|
30
|
+
Requires-Dist: numba==0.68.0
|
|
31
|
+
Requires-Dist: numpy==2.5.3
|
|
32
|
+
Requires-Dist: packaging==26.3
|
|
33
|
+
Requires-Dist: pandas==3.0.6
|
|
34
|
+
Requires-Dist: patsy==1.0.3
|
|
35
|
+
Requires-Dist: pyarrow==25.0.1
|
|
36
|
+
Requires-Dist: python-dateutil==2.9.0.post0
|
|
37
|
+
Requires-Dist: requests==2.34.2
|
|
38
|
+
Requires-Dist: scikit-bio==0.7.4
|
|
39
|
+
Requires-Dist: scipy==1.18.1
|
|
40
|
+
Requires-Dist: six==1.17.0
|
|
41
|
+
Requires-Dist: statsmodels==0.15.0
|
|
42
|
+
Requires-Dist: typing-extensions==4.16.0
|
|
43
|
+
Requires-Dist: urllib3==2.8.0
|
|
44
|
+
Requires-Dist: wrapt==2.5.0
|
|
45
|
+
Description-Content-Type: text/markdown
|
|
46
|
+
|
|
47
|
+
# skb-arrow
|
|
48
|
+
|
|
49
|
+
A process host that exposes [scikit-bio](https://scikit.bio) to non-Python callers. A caller
|
|
50
|
+
runs `skb-arrow`, speaks JSON lines on its stdin and stdout, and exchanges tables as Arrow IPC
|
|
51
|
+
in memory-mapped files. Each capability is one curated, typed, versioned scikit-bio operation:
|
|
52
|
+
`ancombc` and `mantel` so far.
|
|
53
|
+
|
|
54
|
+
## Platforms
|
|
55
|
+
- Linux x86_64, glibc 2.28 or later
|
|
56
|
+
- macOS 12 or later, on Apple silicon
|
|
57
|
+
|
|
58
|
+
Python 3.14 only. Every dependency installs from a wheel, and each is pinned exactly, so an
|
|
59
|
+
answer is reproducible per skb-arrow version.
|
|
60
|
+
|
|
61
|
+
## Install
|
|
62
|
+
Install [uv](https://docs.astral.sh/uv/getting-started/installation/) 0.6.8 or later, then:
|
|
63
|
+
|
|
64
|
+
```
|
|
65
|
+
uv tool install --managed-python --python 3.14 skb-arrow
|
|
66
|
+
uv tool update-shell # if uv says its tool directory isn't on PATH; then open a new shell
|
|
67
|
+
skb-arrow --doctor
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
- `--python 3.14`: uv 0.10 and later skip a default Python older than 3.14, but not a newer
|
|
71
|
+
one: uv ignores the upper bound of the versions skb-arrow supports.
|
|
72
|
+
- `--managed-python`: uv's own CPython, not whichever 3.14 the machine has.
|
|
73
|
+
|
|
74
|
+
`skb-arrow --doctor` reports the interpreter, the platform, the pinned dependencies, numpy's
|
|
75
|
+
BLAS, the thread settings, and where segments go, then a `problem:` line per fault. It exits 1
|
|
76
|
+
if there is one.
|
|
77
|
+
|
|
78
|
+
## Use
|
|
79
|
+
A caller runs `skb-arrow --segment-dir DIR`. `skb-arrow --version` prints the version, the
|
|
80
|
+
protocol version, and each capability with its schema version.
|
|
81
|
+
- [Protocol](https://github.com/the-miint/skb-arrow/blob/main/docs/protocol.md): the control
|
|
82
|
+
channel, the handshake, and versioning.
|
|
83
|
+
- [Transport](https://github.com/the-miint/skb-arrow/blob/main/docs/transport.md): segments,
|
|
84
|
+
and where DIR goes.
|
|
85
|
+
- [Capabilities](https://github.com/the-miint/skb-arrow/blob/main/docs/capabilities.md): each
|
|
86
|
+
capability's inputs, params, and output.
|
|
87
|
+
- [Errors](https://github.com/the-miint/skb-arrow/blob/main/docs/errors.md): error kinds and
|
|
88
|
+
warnings.
|
|
89
|
+
|
|
90
|
+
These follow `main`. A release's own are under its tag:
|
|
91
|
+
`https://github.com/the-miint/skb-arrow/tree/v<version>/docs`.
|
|
92
|
+
|
|
93
|
+
## Threads
|
|
94
|
+
Each is read from the environment, and `--doctor` shows it:
|
|
95
|
+
- `NUMBA_NUM_THREADS`: numba's kernels (`mantel`); every core by default. Answers don't depend
|
|
96
|
+
on it.
|
|
97
|
+
- `OPENBLAS_NUM_THREADS` for OpenBLAS (Linux, and macOS 12 and 13), `VECLIB_MAXIMUM_THREADS`
|
|
98
|
+
for Accelerate (macOS 14 and later): numpy's and scipy's BLAS, which `--doctor` names.
|
|
99
|
+
Answers are bit for bit on one machine at one count; across them, to floating-point
|
|
100
|
+
tolerance.
|
|
101
|
+
- `OMP_NUM_THREADS`: OpenMP's.
|
|
102
|
+
|
|
103
|
+
## Containers
|
|
104
|
+
On Linux, DIR lives in `/dev/shm`. Docker's default 64 MiB `/dev/shm` holds less than one
|
|
105
|
+
default 256 MiB segment, and a call's inputs and outputs share DIR: run the container with a
|
|
106
|
+
larger `--shm-size`, or on Kubernetes mount an `emptyDir` with `medium: Memory` at `/dev/shm`.
|
|
107
|
+
|
|
108
|
+
## License
|
|
109
|
+
BSD-3-Clause.
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# skb-arrow
|
|
2
|
+
|
|
3
|
+
A process host that exposes [scikit-bio](https://scikit.bio) to non-Python callers. A caller
|
|
4
|
+
runs `skb-arrow`, speaks JSON lines on its stdin and stdout, and exchanges tables as Arrow IPC
|
|
5
|
+
in memory-mapped files. Each capability is one curated, typed, versioned scikit-bio operation:
|
|
6
|
+
`ancombc` and `mantel` so far.
|
|
7
|
+
|
|
8
|
+
## Platforms
|
|
9
|
+
- Linux x86_64, glibc 2.28 or later
|
|
10
|
+
- macOS 12 or later, on Apple silicon
|
|
11
|
+
|
|
12
|
+
Python 3.14 only. Every dependency installs from a wheel, and each is pinned exactly, so an
|
|
13
|
+
answer is reproducible per skb-arrow version.
|
|
14
|
+
|
|
15
|
+
## Install
|
|
16
|
+
Install [uv](https://docs.astral.sh/uv/getting-started/installation/) 0.6.8 or later, then:
|
|
17
|
+
|
|
18
|
+
```
|
|
19
|
+
uv tool install --managed-python --python 3.14 skb-arrow
|
|
20
|
+
uv tool update-shell # if uv says its tool directory isn't on PATH; then open a new shell
|
|
21
|
+
skb-arrow --doctor
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
- `--python 3.14`: uv 0.10 and later skip a default Python older than 3.14, but not a newer
|
|
25
|
+
one: uv ignores the upper bound of the versions skb-arrow supports.
|
|
26
|
+
- `--managed-python`: uv's own CPython, not whichever 3.14 the machine has.
|
|
27
|
+
|
|
28
|
+
`skb-arrow --doctor` reports the interpreter, the platform, the pinned dependencies, numpy's
|
|
29
|
+
BLAS, the thread settings, and where segments go, then a `problem:` line per fault. It exits 1
|
|
30
|
+
if there is one.
|
|
31
|
+
|
|
32
|
+
## Use
|
|
33
|
+
A caller runs `skb-arrow --segment-dir DIR`. `skb-arrow --version` prints the version, the
|
|
34
|
+
protocol version, and each capability with its schema version.
|
|
35
|
+
- [Protocol](https://github.com/the-miint/skb-arrow/blob/main/docs/protocol.md): the control
|
|
36
|
+
channel, the handshake, and versioning.
|
|
37
|
+
- [Transport](https://github.com/the-miint/skb-arrow/blob/main/docs/transport.md): segments,
|
|
38
|
+
and where DIR goes.
|
|
39
|
+
- [Capabilities](https://github.com/the-miint/skb-arrow/blob/main/docs/capabilities.md): each
|
|
40
|
+
capability's inputs, params, and output.
|
|
41
|
+
- [Errors](https://github.com/the-miint/skb-arrow/blob/main/docs/errors.md): error kinds and
|
|
42
|
+
warnings.
|
|
43
|
+
|
|
44
|
+
These follow `main`. A release's own are under its tag:
|
|
45
|
+
`https://github.com/the-miint/skb-arrow/tree/v<version>/docs`.
|
|
46
|
+
|
|
47
|
+
## Threads
|
|
48
|
+
Each is read from the environment, and `--doctor` shows it:
|
|
49
|
+
- `NUMBA_NUM_THREADS`: numba's kernels (`mantel`); every core by default. Answers don't depend
|
|
50
|
+
on it.
|
|
51
|
+
- `OPENBLAS_NUM_THREADS` for OpenBLAS (Linux, and macOS 12 and 13), `VECLIB_MAXIMUM_THREADS`
|
|
52
|
+
for Accelerate (macOS 14 and later): numpy's and scipy's BLAS, which `--doctor` names.
|
|
53
|
+
Answers are bit for bit on one machine at one count; across them, to floating-point
|
|
54
|
+
tolerance.
|
|
55
|
+
- `OMP_NUM_THREADS`: OpenMP's.
|
|
56
|
+
|
|
57
|
+
## Containers
|
|
58
|
+
On Linux, DIR lives in `/dev/shm`. Docker's default 64 MiB `/dev/shm` holds less than one
|
|
59
|
+
default 256 MiB segment, and a call's inputs and outputs share DIR: run the container with a
|
|
60
|
+
larger `--shm-size`, or on Kubernetes mount an `emptyDir` with `medium: Memory` at `/dev/shm`.
|
|
61
|
+
|
|
62
|
+
## License
|
|
63
|
+
BSD-3-Clause.
|