seqtree 0.4.0__tar.gz → 0.6.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {seqtree-0.4.0 → seqtree-0.6.0}/CHANGELOG.md +49 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/CMakeLists.txt +1 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/PKG-INFO +13 -1
- {seqtree-0.4.0 → seqtree-0.6.0}/README.md +12 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/SOURCES.md +7 -1
- {seqtree-0.4.0 → seqtree-0.6.0}/include/seqtree/seqtree.hpp +20 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/pyproject.toml +1 -1
- {seqtree-0.4.0 → seqtree-0.6.0}/python/seqtree/__init__.py +2 -1
- seqtree-0.6.0/python/seqtree/distance.py +119 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/_bindings.cpp +39 -0
- seqtree-0.6.0/src/distance.cpp +111 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/structural.inc +4 -4
- {seqtree-0.4.0 → seqtree-0.6.0}/.gitattributes +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/.gitignore +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/LICENSE +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/ROADMAP.md +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/appendix/.gitignore +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/appendix/.latexmkrc +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/appendix/Makefile +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/appendix/epitope_detection.pdf +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/appendix/evalue.pdf +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/appendix/evalue.tex +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/appendix/evalue_matrix.pdf +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/appendix/mhc1_rocpr.pdf +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/appendix/mhc2_rocpr.pdf +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/appendix/refs.bib +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/examples/01_gapped_search.py +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/examples/02_sequence_dendrogram.py +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/examples/03_indel_positions.py +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/examples/04_island_profile.py +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/include/seqtree/kmer_index.hpp +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/include/seqtree/types.hpp +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/python/seqtree/control.py +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/python/seqtree/data/control_human_trb_aa.txt.gz +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/python/seqtree/evalue.py +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/python/seqtree/gapblock.py +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/python/seqtree/layout.py +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/python/seqtree/pairwise.py +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/python/seqtree/pmhc.py +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/python/seqtree/pmhc_evalue.py +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/python/seqtree/py.typed +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/python/seqtree/seeds.py +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/skills/seqtree/SKILL.md +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/atomic_write.hpp +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/blosum45.inc +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/blosum62.inc +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/blosum80.inc +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/codec.cpp +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/engine_seqtm.cpp +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/engine_seqtrie.cpp +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/engines.hpp +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/gapblock.cpp +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/index.cpp +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/kmer_index.cpp +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/pairwise.cpp +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/pam100.inc +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/pam250.inc +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/positional_matrix.cpp +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/searcher.cpp +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/substitution_matrix.cpp +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/trie.cpp +0 -0
- {seqtree-0.4.0 → seqtree-0.6.0}/src/trie.hpp +0 -0
|
@@ -3,6 +3,55 @@
|
|
|
3
3
|
All notable changes to `seqtree`. Dates are release dates; the project is pre-1.0, so a **minor**
|
|
4
4
|
bump may carry breaking changes.
|
|
5
5
|
|
|
6
|
+
## [0.6.0] — 2026-07-17
|
|
7
|
+
|
|
8
|
+
### Fixed
|
|
9
|
+
|
|
10
|
+
- **`structural` scored A–N contacts as if they did not interact, because the source table is
|
|
11
|
+
corrupted there.** The Miyazawa–Jernigan A–N contact energy was transcribed as `0.00` — the
|
|
12
|
+
generator's comment excused it as a pair the source left unlisted. It is not unlisted. In
|
|
13
|
+
`MJ_Keskin_potentials.csv` the lower triangle runs `A-A`, `R-A`, `R-R`, `[N-A]`, `N-R`, …, and
|
|
14
|
+
the `N-A` slot reads `V,1` (mirrored `1,V`), where `1` is a mangled residue symbol. The true
|
|
15
|
+
value is **0.15**. It is not a stray duplicate of `V-N` either — the V row separately lists
|
|
16
|
+
`N = 0.12`.
|
|
17
|
+
|
|
18
|
+
Substituting `0.00` for `0.15` understated A's interaction strength and overstated N's
|
|
19
|
+
(`q(A)` −0.04100 → −0.03350, `q(N)` +0.04350 → +0.05100). Since `structural` is rank-1 by
|
|
20
|
+
construction — every cell is a function of the 20 per-residue strengths — this moved the
|
|
21
|
+
strong→weak ordering that the matrix exists to encode, swapping N and P:
|
|
22
|
+
`FWCLYMIVHGAT`**`NP`**`RSQDEK` → `FWCLYMIVHGAT`**`PN`**`RSQDEK`.
|
|
23
|
+
|
|
24
|
+
Four cells of the 24×24 grid change: `A-W`/`W-A` 6 → 5, and `B-C`/`C-B` 4 → 3 (`B` is
|
|
25
|
+
`mean(N, D)`, so N's shift carries into it). **Scores from `structural()` change for sequences
|
|
26
|
+
containing A, W, or B**; all other matrices are untouched. N and P were near-tied, which is why
|
|
27
|
+
a real correction to the ordering moves so few cells.
|
|
28
|
+
|
|
29
|
+
- **The same comment claimed the source is "near- but not perfectly symmetric".** It is perfectly
|
|
30
|
+
symmetric — 0 asymmetric directed pairs across all 400. The symmetrisation in
|
|
31
|
+
`structural_grid()` is a no-op guard, not a repair, and is now documented as such.
|
|
32
|
+
|
|
33
|
+
## [0.5.0] — 2026-07-16
|
|
34
|
+
|
|
35
|
+
### Added
|
|
36
|
+
|
|
37
|
+
- **`seqtree.distance` — plain Hamming and Levenshtein distances, in C++, without a dependency.**
|
|
38
|
+
The *unweighted* corner of the library: unit costs, no substitution matrix, no gap model, no
|
|
39
|
+
alphabet. When all you need is "how many edits apart are these two strings", you should not have
|
|
40
|
+
to build a `SubstitutionMatrix` or add `python-Levenshtein` / `rapidfuzz`. seqtree still needs
|
|
41
|
+
nothing at runtime.
|
|
42
|
+
|
|
43
|
+
| | |
|
|
44
|
+
|---|---|
|
|
45
|
+
| `distance.hamming(a, b)` | differing positions; **equal length only** (raises `ValueError` otherwise) |
|
|
46
|
+
| `distance.levenshtein(a, b)` | insertions + deletions + substitutions, each cost 1, `O(min(m,n))` memory |
|
|
47
|
+
| `distance.hamming_matrix(a, b, threads=0)` | dense `len(a) × len(b)` int32, GIL released, zero-copy numpy |
|
|
48
|
+
| `distance.levenshtein_matrix(a, b, threads=0)` | same, for mixed-length sequences |
|
|
49
|
+
|
|
50
|
+
Comparison is **case-sensitive**, byte for byte — the one place the library does not fold case,
|
|
51
|
+
since a generic string distance should report the difference it is asked about. For a *weighted*
|
|
52
|
+
alignment (a matrix, affine gaps, local mode) use `seqtree.pairwise` instead. Checked against
|
|
53
|
+
pure-Python oracles over random data in `tests/python/test_distance.py`.
|
|
54
|
+
|
|
6
55
|
## [0.4.0] — 2026-07-11
|
|
7
56
|
|
|
8
57
|
### Added
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.2
|
|
2
2
|
Name: seqtree
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.6.0
|
|
4
4
|
Summary: Fast fuzzy search over biological sequences (C++ core, Python bindings)
|
|
5
5
|
Keywords: sequence-search,fuzzy-matching,CDR3,immunology,bioinformatics,trie
|
|
6
6
|
Author-Email: ISALGO laboratory <mikhail.shugay@gmail.com>
|
|
@@ -89,6 +89,12 @@ Beyond search, seqtree ships:
|
|
|
89
89
|
disagreements** — and **65–87× faster**, since there is no Python in the per-pair loop.
|
|
90
90
|
`dist_matrix` gives `d = s(a,a) + s(b,b) − 2·s(a,b)` directly. BioPython is a *test-only*
|
|
91
91
|
dependency; seqtree still needs nothing at runtime.
|
|
92
|
+
- **Plain edit distances** — `seqtree.distance` is unweighted `hamming` and `levenshtein` (unit
|
|
93
|
+
costs, no matrix, no alphabet) for when you just need a number, not a scored alignment.
|
|
94
|
+
`hamming_matrix` / `levenshtein_matrix` score a whole set against a whole set in one GIL-released
|
|
95
|
+
C++ call (`numpy.asarray` wraps the result with no copy) — no `python-Levenshtein` or `rapidfuzz`
|
|
96
|
+
dependency needed. Hamming requires equal lengths (it raises otherwise); comparison is
|
|
97
|
+
case-sensitive.
|
|
92
98
|
- **Island profiles** — `IslandProfile.fit` builds a position weight matrix over a set of
|
|
93
99
|
frame-aligned junctions (an *island*) and scores a query column by column against the island
|
|
94
100
|
consensus, as a non-negative penalty that flows through `threshold_for_evalue` unchanged. At a
|
|
@@ -181,6 +187,12 @@ score("AAA", "AAAAA", mat, gap_open=5, gap_extend=5) # linear gaps: op
|
|
|
181
187
|
aln = align("CASSLGQAYEQYF", "CASSPGQAYEQF", mat) # + aligned strings and ops
|
|
182
188
|
|
|
183
189
|
d = np.asarray(dist_matrix(v_genes, v_genes, mat, threads=0)) # s(a,a)+s(b,b)-2s(a,b), zero diagonal
|
|
190
|
+
|
|
191
|
+
# plain edit distances -- unweighted Hamming / Levenshtein, no matrix, no dependency
|
|
192
|
+
from seqtree.distance import hamming, levenshtein, hamming_matrix, levenshtein_matrix
|
|
193
|
+
hamming("CASSLGQYF", "CASSPGQYF") # 1 (equal length only)
|
|
194
|
+
levenshtein("kitten", "sitting") # 3
|
|
195
|
+
h = np.asarray(hamming_matrix(umis, umis, threads=0)) # (len, len) int32, zero-copy
|
|
184
196
|
```
|
|
185
197
|
|
|
186
198
|
## Tests
|
|
@@ -55,6 +55,12 @@ Beyond search, seqtree ships:
|
|
|
55
55
|
disagreements** — and **65–87× faster**, since there is no Python in the per-pair loop.
|
|
56
56
|
`dist_matrix` gives `d = s(a,a) + s(b,b) − 2·s(a,b)` directly. BioPython is a *test-only*
|
|
57
57
|
dependency; seqtree still needs nothing at runtime.
|
|
58
|
+
- **Plain edit distances** — `seqtree.distance` is unweighted `hamming` and `levenshtein` (unit
|
|
59
|
+
costs, no matrix, no alphabet) for when you just need a number, not a scored alignment.
|
|
60
|
+
`hamming_matrix` / `levenshtein_matrix` score a whole set against a whole set in one GIL-released
|
|
61
|
+
C++ call (`numpy.asarray` wraps the result with no copy) — no `python-Levenshtein` or `rapidfuzz`
|
|
62
|
+
dependency needed. Hamming requires equal lengths (it raises otherwise); comparison is
|
|
63
|
+
case-sensitive.
|
|
58
64
|
- **Island profiles** — `IslandProfile.fit` builds a position weight matrix over a set of
|
|
59
65
|
frame-aligned junctions (an *island*) and scores a query column by column against the island
|
|
60
66
|
consensus, as a non-negative penalty that flows through `threshold_for_evalue` unchanged. At a
|
|
@@ -147,6 +153,12 @@ score("AAA", "AAAAA", mat, gap_open=5, gap_extend=5) # linear gaps: op
|
|
|
147
153
|
aln = align("CASSLGQAYEQYF", "CASSPGQAYEQF", mat) # + aligned strings and ops
|
|
148
154
|
|
|
149
155
|
d = np.asarray(dist_matrix(v_genes, v_genes, mat, threads=0)) # s(a,a)+s(b,b)-2s(a,b), zero diagonal
|
|
156
|
+
|
|
157
|
+
# plain edit distances -- unweighted Hamming / Levenshtein, no matrix, no dependency
|
|
158
|
+
from seqtree.distance import hamming, levenshtein, hamming_matrix, levenshtein_matrix
|
|
159
|
+
hamming("CASSLGQYF", "CASSPGQYF") # 1 (equal length only)
|
|
160
|
+
levenshtein("kitten", "sitting") # 3
|
|
161
|
+
h = np.asarray(hamming_matrix(umis, umis, threads=0)) # (len, len) int32, zero-copy
|
|
150
162
|
```
|
|
151
163
|
|
|
152
164
|
## Tests
|
|
@@ -32,12 +32,18 @@ possible; and out-of-frame junctions escape thymic selection, making them a samp
|
|
|
32
32
|
| | |
|
|
33
33
|
|---|---|
|
|
34
34
|
| provenance | **derived** (published log-odds / interaction-strength matrices) |
|
|
35
|
-
| origin | BLOSUM62 (NCBI); EMBOSS EPAM250, EPAM100; Miyazawa–Jernigan interaction strengths |
|
|
35
|
+
| origin | BLOSUM62 (NCBI); EMBOSS EPAM250, EPAM100; Miyazawa–Jernigan interaction strengths, transcribed into `MJ_CONTACT` in `bench/gen_matrices.py` from the `MJ` rows of `tcren-ms/src/tcren/data/MJ_Keskin_potentials.csv` (Miyazawa & Jernigan, J Mol Biol 1996, as tabulated by Keskin et al., Protein Sci 1998) |
|
|
36
36
|
| regenerate | `python bench/gen_matrices.py` |
|
|
37
37
|
|
|
38
38
|
Stored as non-negative penalties via the Gram transform `pen[a][b] = s[a][a] + s[b][b] − 2·s[a][b]`,
|
|
39
39
|
clamped at zero, so `penalty(a, a) == 0` and the score defines a ball.
|
|
40
40
|
|
|
41
|
+
**Known corruption in the MJ source.** `MJ_Keskin_potentials.csv` has no `A,N` row: the lower-triangle
|
|
42
|
+
slot where `N-A` belongs is mangled to `V,1` (mirrored `1,V`) for *both* the `MJ` and `Keskin`
|
|
43
|
+
potentials. `MJ_CONTACT` carries the correct `A-N = 0.15` from that row, so the constant will not
|
|
44
|
+
diff cleanly against the CSV at that pair. The `Keskin` rows are unused here. Fix not yet upstreamed
|
|
45
|
+
to `tcren-ms`.
|
|
46
|
+
|
|
41
47
|
## Downloaded on demand
|
|
42
48
|
|
|
43
49
|
### `load_control(name)` → `~/.cache/seqtree/control_*.sqtree`
|
|
@@ -221,4 +221,24 @@ std::vector<int32_t> gapblock_matrix(const std::vector<std::string>& queries,
|
|
|
221
221
|
int32_t gap_extend, const std::vector<int32_t>& prior,
|
|
222
222
|
uint32_t prior_width, int threads = 0);
|
|
223
223
|
|
|
224
|
+
// ---------------------------------------------------------------------------------------
|
|
225
|
+
// Plain string edit distances on raw characters, unit costs. Deliberately alphabet-agnostic:
|
|
226
|
+
// no Codec, no substitution matrix, no gap model -- the distance you want when you just need a
|
|
227
|
+
// number, not a scored alignment. Comparison is byte-for-byte and CASE-SENSITIVE; unlike the
|
|
228
|
+
// search Codec, these do not fold case.
|
|
229
|
+
//
|
|
230
|
+
// hamming -- number of differing positions; defined only for EQUAL-length sequences and
|
|
231
|
+
// throws std::invalid_argument otherwise;
|
|
232
|
+
// levenshtein -- classic insertion/deletion/substitution distance, each edit cost 1,
|
|
233
|
+
// O(min(m,n)) memory.
|
|
234
|
+
int hamming(std::string_view a, std::string_view b);
|
|
235
|
+
int levenshtein(std::string_view a, std::string_view b);
|
|
236
|
+
|
|
237
|
+
// Dense N*K distance matrices, row-major (row i is a[i] vs every b[k]). Parallel;
|
|
238
|
+
// threads <= 0 => hardware_concurrency. hamming_matrix throws on any length-mismatched pair.
|
|
239
|
+
std::vector<int32_t> hamming_matrix(const std::vector<std::string>& a,
|
|
240
|
+
const std::vector<std::string>& b, int threads = 0);
|
|
241
|
+
std::vector<int32_t> levenshtein_matrix(const std::vector<std::string>& a,
|
|
242
|
+
const std::vector<std::string>& b, int threads = 0);
|
|
243
|
+
|
|
224
244
|
} // namespace seqtree
|
|
@@ -24,7 +24,7 @@ from ._core import (
|
|
|
24
24
|
)
|
|
25
25
|
from .control import load_control
|
|
26
26
|
from .evalue import evalues, thetas_from_scores, threshold_for_evalue
|
|
27
|
-
from . import gapblock, layout, pairwise, pmhc, seeds
|
|
27
|
+
from . import distance, gapblock, layout, pairwise, pmhc, seeds
|
|
28
28
|
from .gapblock import (
|
|
29
29
|
GapBlockIndex, IslandProfile, ScoreMatrix, central_prior, embed_in_frame, frame_prior,
|
|
30
30
|
gapblock_score, positions_prior, profile_prior, score_matrix,
|
|
@@ -33,6 +33,7 @@ from .seeds import SeedIndex, core_kmers
|
|
|
33
33
|
from .pmhc import PMHCStore, find_mimics
|
|
34
34
|
|
|
35
35
|
__all__ = [
|
|
36
|
+
"distance",
|
|
36
37
|
"gapblock",
|
|
37
38
|
"layout",
|
|
38
39
|
"pairwise",
|
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
"""Plain string edit distances: Hamming and Levenshtein, in C++, without a dependency.
|
|
2
|
+
|
|
3
|
+
These are the *unweighted* distances -- unit costs, no substitution matrix, no gap model, no
|
|
4
|
+
alphabet. That is the whole point: when all you need is "how many edits apart are these two
|
|
5
|
+
strings", you should not have to build an :class:`~seqtree.SubstitutionMatrix` or reach for
|
|
6
|
+
``python-Levenshtein`` / ``rapidfuzz``. seqtree still needs nothing at runtime.
|
|
7
|
+
|
|
8
|
+
* :func:`hamming` -- number of differing positions; defined only for **equal-length** sequences,
|
|
9
|
+
and raises :class:`ValueError` otherwise;
|
|
10
|
+
* :func:`levenshtein` -- the classic insertion / deletion / substitution edit distance, each edit
|
|
11
|
+
costing 1;
|
|
12
|
+
* :func:`hamming_matrix` / :func:`levenshtein_matrix` -- every ``a`` against every ``b`` in one
|
|
13
|
+
GIL-released, multi-threaded C++ call, returned as a zero-copy :class:`~seqtree.ScoreMatrix`.
|
|
14
|
+
|
|
15
|
+
Comparison is **case-sensitive**, byte for byte -- unlike the search engines, which fold case.
|
|
16
|
+
For a *weighted* alignment (a substitution matrix, affine gaps, local mode), use
|
|
17
|
+
:mod:`seqtree.pairwise` instead.
|
|
18
|
+
|
|
19
|
+
Example:
|
|
20
|
+
>>> from seqtree.distance import hamming, levenshtein
|
|
21
|
+
>>> hamming("CASSLGQYF", "CASSPGQYF")
|
|
22
|
+
1
|
|
23
|
+
>>> levenshtein("kitten", "sitting")
|
|
24
|
+
3
|
|
25
|
+
"""
|
|
26
|
+
from __future__ import annotations
|
|
27
|
+
|
|
28
|
+
from collections.abc import Sequence
|
|
29
|
+
|
|
30
|
+
from ._core import ScoreMatrix
|
|
31
|
+
from ._core import hamming as _hamming
|
|
32
|
+
from ._core import hamming_matrix as _hamming_matrix
|
|
33
|
+
from ._core import levenshtein as _levenshtein
|
|
34
|
+
from ._core import levenshtein_matrix as _levenshtein_matrix
|
|
35
|
+
|
|
36
|
+
__all__ = ["hamming", "levenshtein", "hamming_matrix", "levenshtein_matrix"]
|
|
37
|
+
|
|
38
|
+
|
|
39
|
+
def hamming(a: str, b: str) -> int:
|
|
40
|
+
"""Number of positions at which ``a`` and ``b`` differ.
|
|
41
|
+
|
|
42
|
+
Args:
|
|
43
|
+
a: First sequence.
|
|
44
|
+
b: Second sequence, of the **same length** as ``a``.
|
|
45
|
+
|
|
46
|
+
Returns:
|
|
47
|
+
The count of differing positions (0 when identical).
|
|
48
|
+
|
|
49
|
+
Raises:
|
|
50
|
+
ValueError: If ``a`` and ``b`` have different lengths -- Hamming distance is undefined
|
|
51
|
+
for unequal lengths; use :func:`levenshtein` for that.
|
|
52
|
+
|
|
53
|
+
Example:
|
|
54
|
+
>>> hamming("AAAA", "AAAA")
|
|
55
|
+
0
|
|
56
|
+
>>> hamming("AAAA", "ATAT")
|
|
57
|
+
2
|
|
58
|
+
"""
|
|
59
|
+
return _hamming(a, b)
|
|
60
|
+
|
|
61
|
+
|
|
62
|
+
def levenshtein(a: str, b: str) -> int:
|
|
63
|
+
"""Edit distance: fewest single-character insert / delete / substitute steps from ``a`` to ``b``.
|
|
64
|
+
|
|
65
|
+
Args:
|
|
66
|
+
a: First sequence.
|
|
67
|
+
b: Second sequence; may be any length.
|
|
68
|
+
|
|
69
|
+
Returns:
|
|
70
|
+
The edit distance (0 when identical, ``max(len(a), len(b))`` at most).
|
|
71
|
+
|
|
72
|
+
Example:
|
|
73
|
+
>>> levenshtein("flaw", "lawn")
|
|
74
|
+
2
|
|
75
|
+
>>> levenshtein("CASSLGQAYEQYF", "CASSPGQAYEQF")
|
|
76
|
+
2
|
|
77
|
+
"""
|
|
78
|
+
return _levenshtein(a, b)
|
|
79
|
+
|
|
80
|
+
|
|
81
|
+
def hamming_matrix(a: Sequence[str], b: Sequence[str], threads: int = 0) -> ScoreMatrix:
|
|
82
|
+
"""Hamming distance of every ``a`` against every ``b``, in parallel C++.
|
|
83
|
+
|
|
84
|
+
Args:
|
|
85
|
+
a: Query sequences (the rows).
|
|
86
|
+
b: Reference sequences (the columns).
|
|
87
|
+
threads: Worker threads; ``0`` uses all cores.
|
|
88
|
+
|
|
89
|
+
Returns:
|
|
90
|
+
A :class:`~seqtree.ScoreMatrix` of shape ``(len(a), len(b))`` of int32 distances.
|
|
91
|
+
``numpy.asarray`` wraps it without copying.
|
|
92
|
+
|
|
93
|
+
Raises:
|
|
94
|
+
ValueError: If any ``(a[i], b[k])`` pair has mismatched lengths.
|
|
95
|
+
|
|
96
|
+
Example:
|
|
97
|
+
>>> import numpy as np
|
|
98
|
+
>>> d = np.asarray(hamming_matrix(["AAAA", "AAAT"], ["AAAA", "TTTT"]))
|
|
99
|
+
>>> d.tolist()
|
|
100
|
+
[[0, 4], [1, 3]]
|
|
101
|
+
"""
|
|
102
|
+
return _hamming_matrix(list(a), list(b), threads=threads)
|
|
103
|
+
|
|
104
|
+
|
|
105
|
+
def levenshtein_matrix(a: Sequence[str], b: Sequence[str], threads: int = 0) -> ScoreMatrix:
|
|
106
|
+
"""Levenshtein distance of every ``a`` against every ``b``, in parallel C++.
|
|
107
|
+
|
|
108
|
+
Unlike :func:`hamming_matrix`, sequences may differ in length freely.
|
|
109
|
+
|
|
110
|
+
Args:
|
|
111
|
+
a: Query sequences (the rows).
|
|
112
|
+
b: Reference sequences (the columns).
|
|
113
|
+
threads: Worker threads; ``0`` uses all cores.
|
|
114
|
+
|
|
115
|
+
Returns:
|
|
116
|
+
A :class:`~seqtree.ScoreMatrix` of shape ``(len(a), len(b))`` of int32 distances,
|
|
117
|
+
zero-copy through ``numpy.asarray``.
|
|
118
|
+
"""
|
|
119
|
+
return _levenshtein_matrix(list(a), list(b), threads=threads)
|
|
@@ -575,6 +575,45 @@ PYBIND11_MODULE(_core, m) {
|
|
|
575
575
|
"`prior` is the gap prior flattened to [m][d][i]; see seqtree.gapblock.score_matrix, "
|
|
576
576
|
"which builds it for you.");
|
|
577
577
|
|
|
578
|
+
m.def("hamming", &hamming, py::arg("a"), py::arg("b"),
|
|
579
|
+
"Hamming distance: the number of positions at which two EQUAL-length sequences differ. "
|
|
580
|
+
"Raises ValueError on a length mismatch. Case-sensitive, byte-for-byte.");
|
|
581
|
+
|
|
582
|
+
m.def("levenshtein", &levenshtein, py::arg("a"), py::arg("b"),
|
|
583
|
+
"Levenshtein (edit) distance: the minimum number of single-character insertions, "
|
|
584
|
+
"deletions, and substitutions to turn `a` into `b`, each cost 1. Case-sensitive.");
|
|
585
|
+
|
|
586
|
+
m.def(
|
|
587
|
+
"hamming_matrix",
|
|
588
|
+
[](const std::vector<std::string>& a, const std::vector<std::string>& b, int threads) {
|
|
589
|
+
ScoreMatrix out;
|
|
590
|
+
out.rows = a.size();
|
|
591
|
+
out.cols = b.size();
|
|
592
|
+
{
|
|
593
|
+
py::gil_scoped_release release;
|
|
594
|
+
out.data = hamming_matrix(a, b, threads);
|
|
595
|
+
}
|
|
596
|
+
return out;
|
|
597
|
+
},
|
|
598
|
+
py::arg("a"), py::arg("b"), py::arg("threads") = 0,
|
|
599
|
+
"Dense (len(a), len(b)) int32 Hamming-distance matrix, GIL released. Raises ValueError "
|
|
600
|
+
"if any pair has mismatched lengths.");
|
|
601
|
+
|
|
602
|
+
m.def(
|
|
603
|
+
"levenshtein_matrix",
|
|
604
|
+
[](const std::vector<std::string>& a, const std::vector<std::string>& b, int threads) {
|
|
605
|
+
ScoreMatrix out;
|
|
606
|
+
out.rows = a.size();
|
|
607
|
+
out.cols = b.size();
|
|
608
|
+
{
|
|
609
|
+
py::gil_scoped_release release;
|
|
610
|
+
out.data = levenshtein_matrix(a, b, threads);
|
|
611
|
+
}
|
|
612
|
+
return out;
|
|
613
|
+
},
|
|
614
|
+
py::arg("a"), py::arg("b"), py::arg("threads") = 0,
|
|
615
|
+
"Dense (len(a), len(b)) int32 Levenshtein-distance matrix, GIL released.");
|
|
616
|
+
|
|
578
617
|
py::class_<Candidate>(m, "Candidate",
|
|
579
618
|
"A seed-and-gather hit: peptide_id, shared_kmers (distinct query k-mers "
|
|
580
619
|
"that hit it), best_score. Iterable as (peptide_id, shared_kmers, best_score).")
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
// Plain string edit distances -- Hamming and Levenshtein -- on raw characters, with unit
|
|
2
|
+
// costs. Deliberately alphabet-agnostic: no Codec, no substitution matrix, no gap model. This
|
|
3
|
+
// is the distance you want when you just need a number, not a scored alignment. Everything
|
|
4
|
+
// else in this library scores against a matrix; these two do not, so they live apart from
|
|
5
|
+
// pairwise.cpp even though the threading plumbing is the same.
|
|
6
|
+
//
|
|
7
|
+
// Comparison is byte-for-byte and case-sensitive -- it does NOT fold case the way the search
|
|
8
|
+
// Codec does, because these are generic string distances, not sequence-search primitives.
|
|
9
|
+
|
|
10
|
+
#include "seqtree/seqtree.hpp"
|
|
11
|
+
|
|
12
|
+
#include <algorithm>
|
|
13
|
+
#include <atomic>
|
|
14
|
+
#include <cstdint>
|
|
15
|
+
#include <mutex>
|
|
16
|
+
#include <stdexcept>
|
|
17
|
+
#include <string>
|
|
18
|
+
#include <thread>
|
|
19
|
+
#include <vector>
|
|
20
|
+
|
|
21
|
+
namespace seqtree {
|
|
22
|
+
namespace {
|
|
23
|
+
|
|
24
|
+
// Shared driver for both N*K matrices. `pair` is the scalar distance applied to every cell;
|
|
25
|
+
// it may throw (Hamming does, on a length mismatch), so the worker catches and the exception
|
|
26
|
+
// is rethrown after the join -- an exception escaping a std::thread entry calls std::terminate
|
|
27
|
+
// and would take the interpreter down with an uncatchable SIGABRT. Same plumbing as
|
|
28
|
+
// pairwise.cpp / Index::search_batch.
|
|
29
|
+
template <class Pair>
|
|
30
|
+
std::vector<int32_t> matrix_impl(const std::vector<std::string>& a,
|
|
31
|
+
const std::vector<std::string>& b, int threads, Pair pair) {
|
|
32
|
+
const size_t N = a.size(), K = b.size();
|
|
33
|
+
std::vector<int32_t> out(N * K);
|
|
34
|
+
if (N == 0 || K == 0) return out;
|
|
35
|
+
|
|
36
|
+
unsigned nt =
|
|
37
|
+
threads > 0 ? unsigned(threads) : std::max(1u, std::thread::hardware_concurrency());
|
|
38
|
+
nt = std::min<unsigned>(nt, std::max<size_t>(1, N));
|
|
39
|
+
|
|
40
|
+
std::atomic<size_t> next{0};
|
|
41
|
+
std::exception_ptr err;
|
|
42
|
+
std::mutex emu;
|
|
43
|
+
|
|
44
|
+
auto worker = [&] {
|
|
45
|
+
for (;;) {
|
|
46
|
+
const size_t i = next.fetch_add(1);
|
|
47
|
+
if (i >= N) break;
|
|
48
|
+
try {
|
|
49
|
+
int32_t* row = out.data() + i * K;
|
|
50
|
+
for (size_t k = 0; k < K; ++k) row[k] = pair(a[i], b[k]);
|
|
51
|
+
} catch (...) {
|
|
52
|
+
std::lock_guard<std::mutex> lk(emu);
|
|
53
|
+
if (!err) err = std::current_exception();
|
|
54
|
+
return;
|
|
55
|
+
}
|
|
56
|
+
}
|
|
57
|
+
};
|
|
58
|
+
|
|
59
|
+
std::vector<std::thread> pool;
|
|
60
|
+
for (unsigned t = 0; t < nt; ++t) pool.emplace_back(worker);
|
|
61
|
+
for (auto& th : pool) th.join();
|
|
62
|
+
if (err) std::rethrow_exception(err);
|
|
63
|
+
return out;
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
} // namespace
|
|
67
|
+
|
|
68
|
+
int hamming(std::string_view a, std::string_view b) {
|
|
69
|
+
if (a.size() != b.size())
|
|
70
|
+
throw std::invalid_argument("seqtree: hamming distance is only defined for equal-length "
|
|
71
|
+
"sequences (got " + std::to_string(a.size()) + " and " +
|
|
72
|
+
std::to_string(b.size()) + ")");
|
|
73
|
+
int d = 0;
|
|
74
|
+
for (size_t i = 0; i < a.size(); ++i) d += (a[i] != b[i]);
|
|
75
|
+
return d;
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
int levenshtein(std::string_view a, std::string_view b) {
|
|
79
|
+
// Keep the inner row over the shorter sequence: O(min(m,n)) memory. Two rows, no traceback.
|
|
80
|
+
if (b.size() > a.size()) std::swap(a, b);
|
|
81
|
+
const size_t m = a.size(), n = b.size();
|
|
82
|
+
if (n == 0) return int(m);
|
|
83
|
+
|
|
84
|
+
std::vector<int> prev(n + 1), cur(n + 1);
|
|
85
|
+
for (size_t j = 0; j <= n; ++j) prev[j] = int(j);
|
|
86
|
+
for (size_t i = 1; i <= m; ++i) {
|
|
87
|
+
cur[0] = int(i);
|
|
88
|
+
for (size_t j = 1; j <= n; ++j) {
|
|
89
|
+
const int cost = (a[i - 1] == b[j - 1]) ? 0 : 1;
|
|
90
|
+
cur[j] = std::min({prev[j] + 1, cur[j - 1] + 1, prev[j - 1] + cost});
|
|
91
|
+
}
|
|
92
|
+
prev.swap(cur);
|
|
93
|
+
}
|
|
94
|
+
return prev[n];
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
std::vector<int32_t> hamming_matrix(const std::vector<std::string>& a,
|
|
98
|
+
const std::vector<std::string>& b, int threads) {
|
|
99
|
+
return matrix_impl(a, b, threads, [](std::string_view x, std::string_view y) {
|
|
100
|
+
return hamming(x, y);
|
|
101
|
+
});
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
std::vector<int32_t> levenshtein_matrix(const std::vector<std::string>& a,
|
|
105
|
+
const std::vector<std::string>& b, int threads) {
|
|
106
|
+
return matrix_impl(a, b, threads, [](std::string_view x, std::string_view y) {
|
|
107
|
+
return levenshtein(x, y);
|
|
108
|
+
});
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
} // namespace seqtree
|
|
@@ -4,11 +4,11 @@
|
|
|
4
4
|
static const int32_t kStructuralSize = 24;
|
|
5
5
|
static const int32_t kStructural[24 * 24] = {
|
|
6
6
|
// A R N D C Q E G H I L K M F P S T W Y V B Z X *
|
|
7
|
-
10, 9, 9, 7, 6, 8, 7, 10, 9, 6, 6, 5, 6, 5, 9, 8, 9,
|
|
7
|
+
10, 9, 9, 7, 6, 8, 7, 10, 9, 6, 6, 5, 6, 5, 9, 8, 9, 5, 6, 7, 8, 7, 7, 0,
|
|
8
8
|
9, 10, 10, 8, 4, 9, 8, 8, 8, 5, 4, 6, 5, 4, 10, 10, 9, 4, 5, 6, 9, 9, 7, 0,
|
|
9
9
|
9, 10, 10, 8, 4, 9, 8, 8, 8, 5, 4, 6, 5, 4, 10, 10, 9, 4, 5, 6, 9, 9, 7, 0,
|
|
10
10
|
7, 8, 8, 10, 3, 9, 10, 7, 6, 3, 3, 8, 3, 2, 8, 9, 8, 3, 3, 4, 9, 9, 6, 0,
|
|
11
|
-
6, 4, 4, 3, 10, 4, 2, 6, 7, 9, 10, 1, 9, 9, 4, 4, 5, 10, 10, 8,
|
|
11
|
+
6, 4, 4, 3, 10, 4, 2, 6, 7, 9, 10, 1, 9, 9, 4, 4, 5, 10, 10, 8, 3, 3, 6, 0,
|
|
12
12
|
8, 9, 9, 9, 4, 10, 9, 8, 7, 4, 4, 7, 4, 3, 9, 10, 9, 3, 4, 5, 9, 9, 7, 0,
|
|
13
13
|
7, 8, 8, 10, 2, 9, 10, 6, 6, 3, 2, 8, 3, 2, 8, 8, 7, 2, 3, 4, 9, 9, 6, 0,
|
|
14
14
|
10, 8, 8, 7, 6, 8, 6, 10, 9, 7, 6, 5, 6, 5, 8, 8, 9, 6, 6, 8, 8, 7, 7, 0,
|
|
@@ -21,10 +21,10 @@ static const int32_t kStructural[24 * 24] = {
|
|
|
21
21
|
9, 10, 10, 8, 4, 9, 8, 8, 8, 5, 4, 6, 5, 4, 10, 10, 9, 4, 5, 6, 9, 9, 7, 0,
|
|
22
22
|
8, 10, 10, 9, 4, 10, 8, 8, 7, 5, 4, 6, 5, 4, 10, 10, 9, 4, 4, 6, 9, 9, 7, 0,
|
|
23
23
|
9, 9, 9, 8, 5, 9, 7, 9, 8, 6, 5, 6, 5, 4, 9, 9, 10, 5, 5, 7, 9, 8, 7, 0,
|
|
24
|
-
|
|
24
|
+
5, 4, 4, 3, 10, 3, 2, 6, 6, 9, 10, 0, 9, 10, 4, 4, 5, 10, 9, 8, 3, 3, 6, 0,
|
|
25
25
|
6, 5, 5, 3, 10, 4, 3, 6, 7, 10, 10, 1, 10, 9, 5, 4, 5, 9, 10, 9, 4, 3, 6, 0,
|
|
26
26
|
7, 6, 6, 4, 8, 5, 4, 8, 8, 9, 8, 2, 9, 8, 6, 6, 7, 8, 9, 10, 5, 5, 7, 0,
|
|
27
|
-
8, 9, 9, 9,
|
|
27
|
+
8, 9, 9, 9, 3, 9, 9, 8, 7, 4, 4, 7, 4, 3, 9, 9, 9, 3, 4, 5, 9, 9, 7, 0,
|
|
28
28
|
7, 9, 9, 9, 3, 9, 9, 7, 6, 4, 3, 8, 4, 2, 9, 9, 8, 3, 3, 5, 9, 9, 6, 0,
|
|
29
29
|
7, 7, 7, 6, 6, 7, 6, 7, 7, 7, 6, 4, 7, 6, 7, 7, 7, 6, 6, 7, 7, 6, 7, 0,
|
|
30
30
|
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 10,
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|