cellarr 0.2.2__tar.gz → 0.2.4__tar.gz

{cellarr-0.2.2 → cellarr-0.2.4}/.github/workflows/pypi-publish.yml

@@ -13,9 +13,9 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v4
     - name: Set up Python 3.9
-      uses: actions/setup-python@v2
+      uses: actions/setup-python@v5
       with:
         python-version: 3.9
     - name: Install dependencies

{cellarr-0.2.2 → cellarr-0.2.4}/.github/workflows/pypi-test.yml

@@ -19,9 +19,9 @@ jobs:
 
     name: Python ${{ matrix.python-version }}
     steps:
-    - uses: actions/checkout@v2
+    - uses: actions/checkout@v4
     - name: Setup Python
-      uses: actions/setup-python@v2
+      uses: actions/setup-python@v5
       with:
         python-version: ${{ matrix.python-version }}
         cache: 'pip'

{cellarr-0.2.2 → cellarr-0.2.4}/.gitignore

@@ -52,3 +52,5 @@ MANIFEST
 .venv*/
 .conda*/
 .python-version
+
+lightning_logs*

{cellarr-0.2.2 → cellarr-0.2.4}/CHANGELOG.md

@@ -1,5 +1,10 @@
 # Changelog
 
+## Version 0.2.2
+
+- Provide options to consistently extract cell metadata columns across datasets.
+- Update documentation and tests.
+
 ## Version 0.2.1
 
 * Implement dunder methods `__len__`,  `__repr__` and `__str__` for the `CellArrDatasetSlice` class
@@ -19,6 +24,6 @@ for single cells expression profiles, training labels, and study labels. The dat
 This is the first release of the package to support both creation and access to large
 collection of files based on TileDB.
 
-- Provide a build method to create the tiledb collection from a series of data objects.
+- Provide a build method to create the TileDB collection from a series of data objects.
 - Provides `CellArrDataset` class to query these objects on disk.
 - Implements access and coerce methods to interop with other experimental data packages.

{cellarr-0.2.2 → cellarr-0.2.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: cellarr
-Version: 0.2.2
+Version: 0.2.4
 Summary: TileDB-based array storage for genomics data collections.
 Home-page: https://github.com/BiocPy/cellarr
 Author: Jayaram Kancherla
@@ -57,7 +57,8 @@ Building a `CellArrDataset` generates 4 TileDB files in the specified output dir
 - `cell_metadata`: A TileDB file containing cell metadata including mapping to the samples
 they are tagged with in ``sample_metadata``.
 - A matrix TileDB file named by the `layer_matrix_name` parameter. This allows the package
-to store multiple different matrices, e.g. normalized, scaled for the same cell, gene, sample metadata attributes.
+to store multiple different matrices, e.g. 'counts', 'normalized', 'scaled' for the same cell,
+gene, sample metadata attributes.
 
 The organization is inspired by the [MultiAssayExperiment](https://bioconductor.org/packages/release/bioc/html/MultiAssayExperiment.html) data structure.
 
@@ -77,7 +78,8 @@ import numpy as np
 import tempfile
 from cellarr import build_cellarrdataset, CellArrDataset, MatrixOptions
 
-# Create a temporary directory
+# Create a temporary directory, this is where the
+# output files are created. Pick your location here.
 tempdir = tempfile.mkdtemp()
 
 # Read AnnData objects
@@ -114,6 +116,28 @@ some genes are unmeasured or ordered differently in the original experiments.
 ***Note: The objects to build the `CellArrDataset` are expected to be fairly consistent, especially along the feature dimension.
 if these are `AnnData` or `H5AD`objects, all objects must contain an index (in the `var` slot) specifying the gene symbols.***
 
+#### Optionally provide cell metadata columns
+
+If the cell metadata is inconsistent across datasets, you can provide a list of
+columns to standardize during extraction. Any missing columns will be filled with
+the default value `'NA'`, and their data type should be specified as `'ascii'` in
+`CellMetadataOptions`. For example, this build process will create a TileDB store
+for cell metadata containing the columns `'cellids'` and `'tissue'`. If any dataset
+lacks one of these columns, the missing values will be automatically filled with `'NA'`.
+
+```python
+dataset = build_cellarrdataset(
+    output_path=tempdir,
+    files=[adata1, adata2],
+    matrix_options=MatrixOptions(dtype=np.float32),
+    cell_metadata_options=CellMetadataOptions(
+        column_types={"cellids": "ascii", "tissue": "ascii"}
+    ),
+)
+
+print(dataset)
+```
+
 Check out the [documentation](https://biocpy.github.io/cellarr/tutorial.html) for more details.
 
 ### Query a `CellArrDataset`
@@ -165,6 +189,32 @@ datamodule = DataModule(
 )
 ```
 
+The package also includes a simple autoencoder in the `pytorch-lightning` which makes use of the dataloader. This can be used as a template to create custom architectures and models.
+
+```python
+import pytorch_lightning as pl
+from cellarr.autoencoder import AutoEncoder
+
+autoencoder = AutoEncoder(
+    n_genes=len(datamodule.gene_indices),
+    latent_dim=128,
+    hidden_dim=[1024, 1024, 1024],
+    dropout=0.5,
+    input_dropout=0.4,
+    residual=False,
+)
+
+model_path = "/path/to/model/mymodel/"
+params = {
+    "max_epochs": 500,
+    "logger": True,
+    "log_every_n_steps": 1,
+    "limit_train_batches": 100, # to specify number of batches per epoch
+}
+trainer = pl.Trainer(**params)
+trainer.fit(autoencoder, datamodule=datamodule)
+autoencoder.save_all(model_path=model_path)
+```
 <!-- pyscaffold-notes -->
 
 ## Note

{cellarr-0.2.2 → cellarr-0.2.4}/README.md

@@ -27,7 +27,8 @@ Building a `CellArrDataset` generates 4 TileDB files in the specified output dir
 - `cell_metadata`: A TileDB file containing cell metadata including mapping to the samples
 they are tagged with in ``sample_metadata``.
 - A matrix TileDB file named by the `layer_matrix_name` parameter. This allows the package
-to store multiple different matrices, e.g. normalized, scaled for the same cell, gene, sample metadata attributes.
+to store multiple different matrices, e.g. 'counts', 'normalized', 'scaled' for the same cell,
+gene, sample metadata attributes.
 
 The organization is inspired by the [MultiAssayExperiment](https://bioconductor.org/packages/release/bioc/html/MultiAssayExperiment.html) data structure.
 
@@ -47,7 +48,8 @@ import numpy as np
 import tempfile
 from cellarr import build_cellarrdataset, CellArrDataset, MatrixOptions
 
-# Create a temporary directory
+# Create a temporary directory, this is where the
+# output files are created. Pick your location here.
 tempdir = tempfile.mkdtemp()
 
 # Read AnnData objects
@@ -84,6 +86,28 @@ some genes are unmeasured or ordered differently in the original experiments.
 ***Note: The objects to build the `CellArrDataset` are expected to be fairly consistent, especially along the feature dimension.
 if these are `AnnData` or `H5AD`objects, all objects must contain an index (in the `var` slot) specifying the gene symbols.***
 
+#### Optionally provide cell metadata columns
+
+If the cell metadata is inconsistent across datasets, you can provide a list of
+columns to standardize during extraction. Any missing columns will be filled with
+the default value `'NA'`, and their data type should be specified as `'ascii'` in
+`CellMetadataOptions`. For example, this build process will create a TileDB store
+for cell metadata containing the columns `'cellids'` and `'tissue'`. If any dataset
+lacks one of these columns, the missing values will be automatically filled with `'NA'`.
+
+```python
+dataset = build_cellarrdataset(
+    output_path=tempdir,
+    files=[adata1, adata2],
+    matrix_options=MatrixOptions(dtype=np.float32),
+    cell_metadata_options=CellMetadataOptions(
+        column_types={"cellids": "ascii", "tissue": "ascii"}
+    ),
+)
+
+print(dataset)
+```
+
 Check out the [documentation](https://biocpy.github.io/cellarr/tutorial.html) for more details.
 
 ### Query a `CellArrDataset`
@@ -135,6 +159,32 @@ datamodule = DataModule(
 )
 ```
 
+The package also includes a simple autoencoder in the `pytorch-lightning` which makes use of the dataloader. This can be used as a template to create custom architectures and models.
+
+```python
+import pytorch_lightning as pl
+from cellarr.autoencoder import AutoEncoder
+
+autoencoder = AutoEncoder(
+    n_genes=len(datamodule.gene_indices),
+    latent_dim=128,
+    hidden_dim=[1024, 1024, 1024],
+    dropout=0.5,
+    input_dropout=0.4,
+    residual=False,
+)
+
+model_path = "/path/to/model/mymodel/"
+params = {
+    "max_epochs": 500,
+    "logger": True,
+    "log_every_n_steps": 1,
+    "limit_train_batches": 100, # to specify number of batches per epoch
+}
+trainer = pl.Trainer(**params)
+trainer.fit(autoencoder, datamodule=datamodule)
+autoencoder.save_all(model_path=model_path)
+```
 <!-- pyscaffold-notes -->
 
 ## Note

{cellarr-0.2.2 → cellarr-0.2.4}/docs/tutorial.md

@@ -139,6 +139,28 @@ is automatically created. Hence the sample information provided must match
 the number of input files.
 :::
 
+## Optionally provide cell metadata columns
+
+If the cell metadata is inconsistent across datasets, you can provide a list of
+columns to standardize during extraction. Any missing columns will be filled with
+the default value `'NA'`, and their data type should be specified as `'ascii'` in
+`CellMetadataOptions`. For example, this build process will create a TileDB store
+for cell metadata containing the columns `'cellids'` and `'tissue'`. If any dataset
+lacks one of these columns, the missing values will be automatically filled with `'NA'`.
+
+```python
+dataset = build_cellarrdataset(
+    output_path=tempdir,
+    files=[adata1, adata2],
+    matrix_options=MatrixOptions(dtype=np.float32),
+    cell_metadata_options=CellMetadataOptions(
+        column_types={"cellids": "ascii", "tissue": "ascii"}
+    ),
+)
+
+print(dataset)
+```
+
 # Query a `CellArrDataset`
 
 Users have the option to reuse the `dataset` object retuned when building the dataset or by creating a `CellArrDataset` object by initializing it to the path where the files were created.
@@ -237,6 +259,33 @@ batch = next(iter(dataloader))
 expression, labels, studies = batch
 ```
 
+The dataloader can then be used in training models. The package provides a simple autoencoder to serve as a template for the user's own models.
+
+```python
+import pytorch_lightning as pl
+from cellarr.autoencoder import AutoEncoder
+
+autoencoder = AutoEncoder(
+    n_genes=len(datamodule.gene_indices),
+    latent_dim=128,
+    hidden_dim=[1024, 1024, 1024],
+    dropout=0.5,
+    input_dropout=0.4,
+    residual=False,
+)
+
+model_path = "/path/to/model/mymodel/"
+params = {
+    "max_epochs": 500,
+    "logger": True,
+    "log_every_n_steps": 1,
+    "limit_train_batches": 100, # to specify number of batches per epoch
+}
+trainer = pl.Trainer(**params)
+trainer.fit(autoencoder, datamodule=datamodule)
+autoencoder.save_all(model_path=model_path)
+```
+
 ---
 
 Check out the [documentation](https://biocpy.github.io/cellarr/api/modules.html) for more details.