codeconv 1.0.0__tar.gz

codeconv-1.0.0/CHANGELOG.md

@@ -0,0 +1,49 @@
+# Changelog
+
+## 1.0.0
+
+First public release. Accompanies Perik-Zavodskaia et al., *iScience* **29**, 116824 (2026), and the
+STAR Protocols article describing its use end to end.
+
+### What it does
+
+Reconstructs discrete, individually placed cells carrying absolute whole-transcriptome counts from
+spot-based spatial transcriptomics, with no external single-cell reference, and writes them in the
+10x SpaceRanger layout so the result loads directly into Seurat, scanpy and CellChat.
+
+### Notable behaviour
+
+- **Automated topic-number selection.** `recommend_k()` offers five rules; the default keeps the K
+  values whose held-out perplexity falls in the best 5% of the sweep by rank and returns the largest
+  of them. `step5_ksweep()` prints the recommendation, draws it and the low-perplexity band on the
+  K-sweep figure, accepts a manual `n_topics=` override, and warns when the recommendation lands on
+  `max_k` — which means the sweep was too narrow to bound K from above. The choice is published as
+  `codeconv.recommended_K`, and on the result object as `.k`, with `.summary()` returning the whole
+  sweep as a DataFrame.
+- **Built-in species profiles.** Human and mouse live in the module as `DEFAULT_CONFIG`, so a fresh
+  install runs with nothing else to download. An external `codeconv_config.json` takes precedence
+  when one is passed; `write_default_config()` dumps the built-ins for editing.
+- **Platform detection.** `step1_acquisition_and_anchoring()` identifies the input layout and reports
+  it. 10x Visium and DBiT-seq are read natively, the DBiT-seq channel width being taken from the
+  sample name when it carries one. `PLATFORM_PROFILES` holds the geometry, `SliceData.platform`
+  records what was used, and `platform=` overrides detection. Capture units at or below the size of
+  one cell are deliberately out of scope: there is nothing to deconvolve, and segmentation rather
+  than a mixture model is the appropriate step.
+- **CellChat hand-off.** `cellchat_spatial_factors()` derives the pixel-to-micron ratio and tolerance
+  for a spatial CellChat analysis and reports how many within-spot cell pairs a given interaction
+  range captures — the check that communication inference is invariant to the sub-spot cell
+  placement. `step9_export_results()` records all of it in `run_summary.json`, so the downstream R
+  session reads one file instead of re-deriving unit conversions. The ratio is calibrated from the
+  array's documented spot pitch divided by the pitch measured on the slide, because neither the 55 µm
+  physical spot diameter nor CellChat's 65 µm convention reproduces the real geometry.
+- **Streaming reader with a parse cache.** A multi-gigabyte dense count table is parsed row by row
+  into a sparse matrix and cached beside the source, so later loads take under a second.
+- **Placeholder histology.** `make_dummy_tissue_image()` writes a black canvas with matching scale
+  factors for platforms that ship no registered image, so the export still loads in Seurat.
+
+### Note for users of the pre-release script
+
+The first argument of `step1_acquisition_and_anchoring()` is now `spatial_path`, since the pipeline
+is no longer Visium-only. The old `visium_path=` keyword still works and raises a
+`DeprecationWarning`. `config_path` now defaults to `None` and `species` to `"hs"` on every step that
+takes them.

codeconv-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Roman Perik-Zavodskii
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

codeconv-1.0.0/MANIFEST.in

@@ -0,0 +1,4 @@
+include README.md
+include CHANGELOG.md
+include LICENSE
+include codeconv_config.json

codeconv-1.0.0/PKG-INFO

@@ -0,0 +1,220 @@
+Metadata-Version: 2.4
+Name: codeconv
+Version: 1.0.0
+Summary: Reference-free single-cell-resolution deconvolution of spot-based spatial transcriptomics
+Author: Roman Perik-Zavodskii, Saleh Alrhmoun
+Author-email: Olga Perik-Zavodskaia <perik.zavodskaia@gmail.com>, Sergey Sennikov <sennikov@niikim.ru>
+Maintainer-email: Olga Perik-Zavodskaia <perik.zavodskaia@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/Perik-Zavodskii/CoexpressDeconvolve
+Project-URL: Repository, https://github.com/Perik-Zavodskii/CoexpressDeconvolve
+Project-URL: Publication, https://doi.org/10.1016/j.isci.2026.116824
+Project-URL: Archive, https://doi.org/10.5281/zenodo.21860849
+Project-URL: Issues, https://github.com/Perik-Zavodskii/CoexpressDeconvolve/issues
+Keywords: spatial transcriptomics,deconvolution,single-cell,Visium,DBiT-seq,topic model,bioinformatics
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.21
+Requires-Dist: pandas>=1.3
+Requires-Dist: scipy>=1.7
+Requires-Dist: scikit-learn>=1.0
+Requires-Dist: matplotlib>=3.5
+Requires-Dist: seaborn>=0.11
+Requires-Dist: h5py>=3.1
+Requires-Dist: tqdm>=4.62
+Requires-Dist: umap-learn>=0.5.3
+Provides-Extra: notebook
+Requires-Dist: jupyter>=1.0; extra == "notebook"
+Requires-Dist: ipywidgets>=8.0; extra == "notebook"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: build>=1.0; extra == "dev"
+Requires-Dist: twine>=4.0; extra == "dev"
+Dynamic: license-file
+
+# CoexpressDeconvolve
+
+Spot-based spatial transcriptomics measures whole transcriptomes in place, but every spot pools
+several cells, so a spot profile is a mixture rather than a cell. CoexpressDeconvolve resolves that
+mixture into single-cell-like profiles, without needing a matched single-cell dataset to learn from.
+
+The problem it addresses is that the usual route out of the mixture is a reference atlas, and a
+reference is often unavailable, mismatched to the tissue, or missing the very populations that were
+lost during dissociation. CoexpressDeconvolve works from the slide alone.
+
+Three quantities carry the method. How many cells a spot captured is estimated from the
+housekeeping-gene signal together with the total UMI count. Which expression programmes are present
+is learned de novo from gene co-expression across the whole slide. How the spot's transcripts belong
+to those cells follows from distributing its UMIs among them, conserving the per-spot total exactly:
+the reconstructed cells of a spot add back up to the transcriptome measured there.
+
+Each cell comes out with a whole transcriptome, absolute counts and a coordinate inside its parent
+spot, written in the standard 10x SpaceRanger layout, so the result opens in Seurat or scanpy like
+any other single-cell object.
+
+Method and benchmarks: Perik-Zavodskaia, O., Perik-Zavodskii, R., Alrhmoun, S. & Sennikov, S.
+*iScience* **29**, 116824 (2026). https://doi.org/10.1016/j.isci.2026.116824
+
+## Supported platforms
+
+The platform is detected from the input layout and announced at load. Everything downstream is
+platform-agnostic: Step 1 normalizes every input to counts, coordinates, a capture footprint and a
+pitch.
+
+| Platform | Capture unit | Pitch | Layout expected | Histology |
+|---|---|---|---|---|
+| **10x Visium** (any capture area) | 55 µm | 100 µm | SpaceRanger: `filtered_feature_bc_matrix.h5` (or the `.mtx` folder) + `spatial/` | yes |
+| **DBiT-seq** | 10 / 25 / 50 µm | 2 × channel width | one count table on the 50 × 50 grid, labels `AxB` | no |
+
+The DBiT-seq channel width is read from the sample name when it carries one (`GSM4189611_50t` -> 50 µm),
+and the pitch is twice the width. Platforms that ship no tissue image get a black placeholder canvas
+sized to the capture grid, with synthesized scale factors, so `Seurat::Load10X_Spatial()` and every
+spatial plot still work.
+
+Capture units at or below the size of one cell are out of scope: there is nothing to deconvolve, and
+segmentation rather than a mixture model is the right step there.
+
+## Installation
+
+```bash
+pip install codeconv
+```
+
+Human (`hs`) and mouse (`mm`) species profiles are built into the module, so nothing else has to be
+downloaded to get started.
+
+## Usage
+
+```python
+import codeconv
+codeconv.set_seed(42)
+
+spatial_path  = "./Glioblastoma"
+output_folder = "."
+species       = "hs"                      # "hs" | "mm" | anything in your config
+
+slices   = codeconv.step1_acquisition_and_anchoring(spatial_path)
+slices   = codeconv.step2_estimate_cell_density(slices, species=species,
+                                                min_umi=900, anchor_mean_factor=0.7)
+odg_pack = codeconv.step3_feature_selection(slices, species=species, n_odg=3000)
+manifold = codeconv.step4_gene_manifold(slices, odg_pack, n_components=30)
+
+ksweep   = codeconv.step5_ksweep(odg_pack, min_k=3, max_k=15)
+print(ksweep.summary())  # per-K perplexity, rare topics, which K are in the low band
+
+model = codeconv.step6_final_deconvolution(slices, odg_pack, manifold,
+                                           n_topics=codeconv.recommended_K,  # or your own K
+                                           k_neighbors=3)
+cells = codeconv.step7_sampling_engine(slices, model, species=species)
+codeconv.step8_geometry_and_placement(cells, slices)
+codeconv.step9_export_results(cells, slices, output_folder, interaction_range_um=50.0)
+```
+
+### Several slices at once
+
+Pass a dict instead of a string. Topics are aligned across slices, so the reconstructed cell types
+are comparable between samples:
+
+```python
+spatial_path = {"sample_A": "./A", "sample_B": "./B"}
+```
+
+Per-slice parameters (`min_umi`, `anchor_mean_factor`, `low_slice_quality`) take either a scalar,
+broadcast to every slice, or a dict keyed by slice name:
+
+```python
+slices = codeconv.step2_estimate_cell_density(
+    slices, species="hs",
+    min_umi={"sample_A": 900, "sample_B": 1200},
+    anchor_mean_factor=0.7,
+)
+```
+
+A list of paths also works, with slice names taken from the folder names.
+
+### Choosing K automatically
+
+The number of topics used to be a visual judgement call. It no longer is:
+
+```python
+ksweep = codeconv.step5_ksweep(odg_pack, min_k=3, max_k=15)   # prints the recommendation
+```
+
+The default rule keeps every K whose held-out perplexity lands in the best 5% of the sweep by rank
+and returns the **largest** of them, which is the highest K still in the low-perplexity regime. That
+is deliberate: because CoexpressDeconvolve returns individual cells rather than fractions, a rare
+topic is a minor cell population worth recovering, not evidence of over-splitting. Alternatives:
+
+```python
+codeconv.recommend_k(ksweep.perplexity['Glioblastoma'], rule="relative_tolerance", tol=0.01)
+codeconv.recommend_k(ksweep.perplexity['Glioblastoma'], rule="lowest_perplexity")
+```
+
+A recommendation landing on `max_k` means perplexity had not turned back up yet and the sweep is too
+narrow; you get a warning. Pass `n_topics=` to record a manual choice, and it is drawn on the plot
+next to the automatic one.
+
+### The 9 steps
+
+1. **Acquisition** - detect the platform, load counts and coordinates, per-spot UMI QC.
+2. **Density** - cells per spot from a hybrid housekeeping/UMI calibration. `low_slice_quality=True`
+   enforces a floor of one cell on every spot passing the UMI gate.
+3. **Feature selection** - noise-gene regex, presence filter, overdispersed genes from the
+   mean-variance trend.
+4. **Manifold** - joint gene co-expression topology via ICA + UMAP.
+5. **K-sweep** - held-out perplexity across K, rare-topic counts, automatic recommendation.
+6. **Deconvolution** - per-slice LDA, Hungarian topic alignment across slices, mean-consensus beta,
+   per-slice theta refit against the frozen consensus, projection onto each slice's full gene list.
+7. **Sampling** - discrete cells drawn per spot; per-spot UMI totals are conserved exactly.
+8. **Placement** - cells positioned inside their parent spot footprint by Vogel packing.
+9. **Export** - 10x layout under `output_folder/slice_<name>/deconvolved/`, one folder per slice.
+
+## Downstream analysis
+
+Each slice exports a `filtered_feature_bc_matrix.h5` and a `spatial/` folder. From there the
+reconstruction is an ordinary single-cell spatial object and goes wherever such objects go:
+
+```r
+library(Seurat)
+seurat_obj <- Load10X_Spatial("./slice_Glioblastoma/deconvolved")
+```
+
+```python
+import scanpy as sc
+adata = sc.read_visium("./slice_Glioblastoma/deconvolved")
+```
+
+Clustering, annotation, trajectory inference, cell-cell communication - anything that takes a
+single-cell object works, with no adapters. `Seurat Spatial.ipynb` in the repository is a worked
+example. Topics are aligned across slices, so cluster comparison between samples is meaningful.
+
+## Citation
+
+```bibtex
+@article{PerikZavodskaia2026CoexpressDeconvolve,
+  title   = {CoexpressDeconvolve enables reference-free single-cell-resolution
+             deconvolution from spot-based spatial transcriptomics},
+  author  = {Perik-Zavodskaia, Olga and Perik-Zavodskii, Roman and
+             Alrhmoun, Saleh and Sennikov, Sergey},
+  journal = {iScience},
+  volume  = {29},
+  pages   = {116824},
+  year    = {2026},
+  doi     = {10.1016/j.isci.2026.116824}
+}
+```
+
+## License
+
+MIT, see `LICENSE`.

codeconv-1.0.0/README.md

@@ -0,0 +1,175 @@
+# CoexpressDeconvolve
+
+Spot-based spatial transcriptomics measures whole transcriptomes in place, but every spot pools
+several cells, so a spot profile is a mixture rather than a cell. CoexpressDeconvolve resolves that
+mixture into single-cell-like profiles, without needing a matched single-cell dataset to learn from.
+
+The problem it addresses is that the usual route out of the mixture is a reference atlas, and a
+reference is often unavailable, mismatched to the tissue, or missing the very populations that were
+lost during dissociation. CoexpressDeconvolve works from the slide alone.
+
+Three quantities carry the method. How many cells a spot captured is estimated from the
+housekeeping-gene signal together with the total UMI count. Which expression programmes are present
+is learned de novo from gene co-expression across the whole slide. How the spot's transcripts belong
+to those cells follows from distributing its UMIs among them, conserving the per-spot total exactly:
+the reconstructed cells of a spot add back up to the transcriptome measured there.
+
+Each cell comes out with a whole transcriptome, absolute counts and a coordinate inside its parent
+spot, written in the standard 10x SpaceRanger layout, so the result opens in Seurat or scanpy like
+any other single-cell object.
+
+Method and benchmarks: Perik-Zavodskaia, O., Perik-Zavodskii, R., Alrhmoun, S. & Sennikov, S.
+*iScience* **29**, 116824 (2026). https://doi.org/10.1016/j.isci.2026.116824
+
+## Supported platforms
+
+The platform is detected from the input layout and announced at load. Everything downstream is
+platform-agnostic: Step 1 normalizes every input to counts, coordinates, a capture footprint and a
+pitch.
+
+| Platform | Capture unit | Pitch | Layout expected | Histology |
+|---|---|---|---|---|
+| **10x Visium** (any capture area) | 55 µm | 100 µm | SpaceRanger: `filtered_feature_bc_matrix.h5` (or the `.mtx` folder) + `spatial/` | yes |
+| **DBiT-seq** | 10 / 25 / 50 µm | 2 × channel width | one count table on the 50 × 50 grid, labels `AxB` | no |
+
+The DBiT-seq channel width is read from the sample name when it carries one (`GSM4189611_50t` -> 50 µm),
+and the pitch is twice the width. Platforms that ship no tissue image get a black placeholder canvas
+sized to the capture grid, with synthesized scale factors, so `Seurat::Load10X_Spatial()` and every
+spatial plot still work.
+
+Capture units at or below the size of one cell are out of scope: there is nothing to deconvolve, and
+segmentation rather than a mixture model is the right step there.
+
+## Installation
+
+```bash
+pip install codeconv
+```
+
+Human (`hs`) and mouse (`mm`) species profiles are built into the module, so nothing else has to be
+downloaded to get started.
+
+## Usage
+
+```python
+import codeconv
+codeconv.set_seed(42)
+
+spatial_path  = "./Glioblastoma"
+output_folder = "."
+species       = "hs"                      # "hs" | "mm" | anything in your config
+
+slices   = codeconv.step1_acquisition_and_anchoring(spatial_path)
+slices   = codeconv.step2_estimate_cell_density(slices, species=species,
+                                                min_umi=900, anchor_mean_factor=0.7)
+odg_pack = codeconv.step3_feature_selection(slices, species=species, n_odg=3000)
+manifold = codeconv.step4_gene_manifold(slices, odg_pack, n_components=30)
+
+ksweep   = codeconv.step5_ksweep(odg_pack, min_k=3, max_k=15)
+print(ksweep.summary())  # per-K perplexity, rare topics, which K are in the low band
+
+model = codeconv.step6_final_deconvolution(slices, odg_pack, manifold,
+                                           n_topics=codeconv.recommended_K,  # or your own K
+                                           k_neighbors=3)
+cells = codeconv.step7_sampling_engine(slices, model, species=species)
+codeconv.step8_geometry_and_placement(cells, slices)
+codeconv.step9_export_results(cells, slices, output_folder, interaction_range_um=50.0)
+```
+
+### Several slices at once
+
+Pass a dict instead of a string. Topics are aligned across slices, so the reconstructed cell types
+are comparable between samples:
+
+```python
+spatial_path = {"sample_A": "./A", "sample_B": "./B"}
+```
+
+Per-slice parameters (`min_umi`, `anchor_mean_factor`, `low_slice_quality`) take either a scalar,
+broadcast to every slice, or a dict keyed by slice name:
+
+```python
+slices = codeconv.step2_estimate_cell_density(
+    slices, species="hs",
+    min_umi={"sample_A": 900, "sample_B": 1200},
+    anchor_mean_factor=0.7,
+)
+```
+
+A list of paths also works, with slice names taken from the folder names.
+
+### Choosing K automatically
+
+The number of topics used to be a visual judgement call. It no longer is:
+
+```python
+ksweep = codeconv.step5_ksweep(odg_pack, min_k=3, max_k=15)   # prints the recommendation
+```
+
+The default rule keeps every K whose held-out perplexity lands in the best 5% of the sweep by rank
+and returns the **largest** of them, which is the highest K still in the low-perplexity regime. That
+is deliberate: because CoexpressDeconvolve returns individual cells rather than fractions, a rare
+topic is a minor cell population worth recovering, not evidence of over-splitting. Alternatives:
+
+```python
+codeconv.recommend_k(ksweep.perplexity['Glioblastoma'], rule="relative_tolerance", tol=0.01)
+codeconv.recommend_k(ksweep.perplexity['Glioblastoma'], rule="lowest_perplexity")
+```
+
+A recommendation landing on `max_k` means perplexity had not turned back up yet and the sweep is too
+narrow; you get a warning. Pass `n_topics=` to record a manual choice, and it is drawn on the plot
+next to the automatic one.
+
+### The 9 steps
+
+1. **Acquisition** - detect the platform, load counts and coordinates, per-spot UMI QC.
+2. **Density** - cells per spot from a hybrid housekeeping/UMI calibration. `low_slice_quality=True`
+   enforces a floor of one cell on every spot passing the UMI gate.
+3. **Feature selection** - noise-gene regex, presence filter, overdispersed genes from the
+   mean-variance trend.
+4. **Manifold** - joint gene co-expression topology via ICA + UMAP.
+5. **K-sweep** - held-out perplexity across K, rare-topic counts, automatic recommendation.
+6. **Deconvolution** - per-slice LDA, Hungarian topic alignment across slices, mean-consensus beta,
+   per-slice theta refit against the frozen consensus, projection onto each slice's full gene list.
+7. **Sampling** - discrete cells drawn per spot; per-spot UMI totals are conserved exactly.
+8. **Placement** - cells positioned inside their parent spot footprint by Vogel packing.
+9. **Export** - 10x layout under `output_folder/slice_<name>/deconvolved/`, one folder per slice.
+
+## Downstream analysis
+
+Each slice exports a `filtered_feature_bc_matrix.h5` and a `spatial/` folder. From there the
+reconstruction is an ordinary single-cell spatial object and goes wherever such objects go:
+
+```r
+library(Seurat)
+seurat_obj <- Load10X_Spatial("./slice_Glioblastoma/deconvolved")
+```
+
+```python
+import scanpy as sc
+adata = sc.read_visium("./slice_Glioblastoma/deconvolved")
+```
+
+Clustering, annotation, trajectory inference, cell-cell communication - anything that takes a
+single-cell object works, with no adapters. `Seurat Spatial.ipynb` in the repository is a worked
+example. Topics are aligned across slices, so cluster comparison between samples is meaningful.
+
+## Citation
+
+```bibtex
+@article{PerikZavodskaia2026CoexpressDeconvolve,
+  title   = {CoexpressDeconvolve enables reference-free single-cell-resolution
+             deconvolution from spot-based spatial transcriptomics},
+  author  = {Perik-Zavodskaia, Olga and Perik-Zavodskii, Roman and
+             Alrhmoun, Saleh and Sennikov, Sergey},
+  journal = {iScience},
+  volume  = {29},
+  pages   = {116824},
+  year    = {2026},
+  doi     = {10.1016/j.isci.2026.116824}
+}
+```
+
+## License
+
+MIT, see `LICENSE`.