cap-anndata 0.1.0__tar.gz

cap_anndata-0.1.0/LICENSE

@@ -0,0 +1,28 @@
+BSD 3-Clause License
+
+Copyright (c) 2024, R. Mukhin, A. Isaev, Cell-Annotation Platform
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

cap_anndata-0.1.0/PKG-INFO

@@ -0,0 +1,199 @@
+Metadata-Version: 2.1
+Name: cap_anndata
+Version: 0.1.0
+Summary: Partial read of AnnData files for low-memory operations with large datasets.
+Home-page: https://github.com/cellannotation/cap-anndata
+Author: R. Mukhin, A. Isaev
+Author-email: roman@ebookapplications.com
+Project-URL: Bug Tracker, https://github.com/cellannotation/cap-anndata/issues
+Classifier: Programming Language :: Python :: 3.9
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.26.3
+Requires-Dist: pandas>=2.2.0
+Requires-Dist: anndata>=0.10.5
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0.0; extra == "dev"
+
+# CAP-AnnData: Enhanced Partial I/O for AnnData Files
+
+## Overview
+CAP-AnnData enriches the AnnData ecosystem by offering tailored functionalities for partial reading and writing of AnnData files. This enhancement allows for selective manipulation of sections such as `obs`, `var`, `X`, `raw.X`, `obsm`, and `uns` without the need for loading entire datasets into memory. Leveraging AnnData's native methods, CAP-AnnData aims to maintain backward compatibility while improving efficiency, especially useful for large-scale single-cell genomics data.
+
+## Getting Started
+
+### Running Tests
+Ensure the integrity and reliability of CAP-AnnData on your system by running the unit tests in `test/unit_test.py`.
+
+Make sure Python 3.9 or newer is used, along with all requirements specified in requirements.txt
+
+## How-TO:
+
+#### 1. Read AnnData File Dataframes
+
+##### Basic Reading
+By default, `CapAnnData` does not automatically read any data. To begin working with dataframes, you need to explicitly read the data from the AnnData file. You can read the entire dataframe or select specific columns. For partial reading, provide a list of column names.
+
+```python
+import h5py
+from cap_anndata import CapAnnData
+
+file_path = "your_data.h5ad"
+with h5py.File(file_path, 'r') as file:
+    cap_adata = CapAnnData(file)
+
+    # Read all columns of 'obs'
+    cap_adata.read_obs()
+
+    # Read specific columns of 'var'
+    cap_adata.read_var(columns=['gene_expression', 'dispersion'])
+
+    # Read all columns of raw.var
+    cap_adata.read_var(raw=True)
+```
+
+##### Non-existing columns
+
+If a column doesn't exist in the file, no error will be raised but the column will be missing in the resulting Dataframe. So, the list of columns saying more like "try to read this columns from the file". It is needed because we there is no way yet to check if the column exists before the read. 
+
+#### 2. Modify the AnnData File Dataframes In-Place
+
+You can directly modify the dataframe by adding, renaming, or removing columns.
+
+```python
+# Create a new column
+cap_adata.obs['new_col'] = [value1, value2, value3]
+
+# Rename a column
+cap_adata.rename_column('old_col_name', 'new_col_name')
+
+# Remove a column
+cap_adata.remove_column('col_to_remove')
+```
+
+After modifications, you can overwrite the changes back to the AnnData file. If a value doesn't exist, it will be created.
+
+```python
+# overwrite all values which were read
+cap_adata.overwrite()
+
+# overwrite choosen fields
+cap_adata.overwrite(['obs', 'var'])
+```
+
+The full list of supported fields: `X`, `raw.X`, `obs`, `var`, `raw.var`, `obsm`, `uns`.
+
+#### 3. How to Read Few Columns but Overwrite One in a Dataframe
+
+The only way yet to do that is to drop all columns from in-memory dataframe (with `pandas.drop`!) before the call of `overwrite` method.
+
+```python
+# Read specific columns
+cap_adata.read_obs(columns=['cell_type', 'sample'])
+
+# Drop a column in-memory
+# DON'T USE remove_column here!
+cap_adata.obs.drop(columns='sample', inplace=True)
+
+# Overwrite changes
+cap_adata.overwrite(['obs'])
+```
+
+#### 4. How to work with X and raw.X
+
+The CapAnnData package won't read any field by default. However, the `X` and `raw.X` will be linked to the backed matrices automatically upon the first request to those fields.
+
+```python
+with h5py.File(path) as file:
+    # self.X is None here
+    cap_adata = CapAnnData(file)  
+
+    # will return the h5py.Dataset or CSRDataset
+    x = cap_adata.X  
+
+    # The same for raw.X
+    raw_x = cap_adata.raw.X 
+
+    # take whole matrix in memory
+    x = cap_adata.X[:] 
+```
+
+The CapAnnData supports the standard `numpy`/`h5py` sclising rules
+
+```python
+# slice rows
+s_ = np.s_[0:5]
+# slice columns
+s_ = np.s_[:, 0:5]
+# boolean mask + slicing
+mask = np.array([i < 5 for i in range(adata.shape[0])])
+s_ = np.s_[mask, :5]
+```
+
+#### 5. How to handle obsm embeddings matrixes
+
+By the default the CapAnnData will not read the embeddings matrix. The link to the h5py objects will be created upon the first call of the `.obsm` property. Alike the AnnData package the call like `cap_adata.obsm["X_tsne"]` will not return the in-memory matrix but will return the backed version instead. We can get the information about the name and shape of the embeddings without taking the whole matrixes in the memory!
+
+```python
+with h5py.File(path) as file:
+    # initialization
+    cap_adata = CapAnnData(file) 
+
+    # will return the list of strings
+    obsm_keys = cap_adata.obsm_keys()  
+
+    # return the shape of the matrix in backed mode
+    embeddings = obsm_keys[0]
+    shape = cap_adata.obsm[embeddings].shape  
+
+    # take the whole matrix in memory
+    matrix = cap_adata.obsm[embeddings][:]
+```
+
+#### 6. How to read and modify uns section
+
+The `CapAnnData` class will lazely link the uns section upon the first call but ***WILL NOT*** read it into memory. Instead, the dictionary of the pairs `{'key': "__NotLinkedObject"}` will be creted. It allow to get the list of keys before the actual read. To read the uns section in the memory the `.read_uns(keys)` method must be called.
+
+```python
+with h5py.File(path) as file:
+    # initialization
+    cap_adata = CapAnnData(file) 
+
+    # will return the keys() object
+    keys = cap_adata.uns.keys()  
+
+    # read in memory the first key only
+    cap_adata.read_uns([keys[0]])
+
+    # read the whole uns section into memory
+    cap_adata.read_uns()
+```
+
+Since the `.uns` section is in the memory (partially or completely) we can work with it as with the regular `dict()` python object. The main feature of the `CapAnnDataUns` class which inherited from `dict` is the tracking of the keys which must be removed from the `.h5ad` file upon overwrite. 
+
+```python
+# get the value
+v = cap_adata.uns["key1"]
+v = cap_adata.uns.get("key1")
+
+# modify values
+cap_adata.uns["key1"] = "new_value"
+
+# create new keys
+cap_adata.uns["new_key"] = "value"
+
+# remove keys
+cap_adata.uns.pop("key1")  # is recommended way
+del cap_adata.uns.pop("key2")
+cap_adata.uns.popitem()
+```
+
+To save `uns` changes the method `CapAnnData.overwrite()` must be called. 
+
+```python
+cap_adata.overwrite()  # all in-memory fields will be overwritten
+cap_adata.overwrite(["uns"])  # overwrite the uns secion only
+```

cap_anndata-0.1.0/README.md

@@ -0,0 +1,179 @@
+# CAP-AnnData: Enhanced Partial I/O for AnnData Files
+
+## Overview
+CAP-AnnData enriches the AnnData ecosystem by offering tailored functionalities for partial reading and writing of AnnData files. This enhancement allows for selective manipulation of sections such as `obs`, `var`, `X`, `raw.X`, `obsm`, and `uns` without the need for loading entire datasets into memory. Leveraging AnnData's native methods, CAP-AnnData aims to maintain backward compatibility while improving efficiency, especially useful for large-scale single-cell genomics data.
+
+## Getting Started
+
+### Running Tests
+Ensure the integrity and reliability of CAP-AnnData on your system by running the unit tests in `test/unit_test.py`.
+
+Make sure Python 3.9 or newer is used, along with all requirements specified in requirements.txt
+
+## How-TO:
+
+#### 1. Read AnnData File Dataframes
+
+##### Basic Reading
+By default, `CapAnnData` does not automatically read any data. To begin working with dataframes, you need to explicitly read the data from the AnnData file. You can read the entire dataframe or select specific columns. For partial reading, provide a list of column names.
+
+```python
+import h5py
+from cap_anndata import CapAnnData
+
+file_path = "your_data.h5ad"
+with h5py.File(file_path, 'r') as file:
+    cap_adata = CapAnnData(file)
+
+    # Read all columns of 'obs'
+    cap_adata.read_obs()
+
+    # Read specific columns of 'var'
+    cap_adata.read_var(columns=['gene_expression', 'dispersion'])
+
+    # Read all columns of raw.var
+    cap_adata.read_var(raw=True)
+```
+
+##### Non-existing columns
+
+If a column doesn't exist in the file, no error will be raised but the column will be missing in the resulting Dataframe. So, the list of columns saying more like "try to read this columns from the file". It is needed because we there is no way yet to check if the column exists before the read. 
+
+#### 2. Modify the AnnData File Dataframes In-Place
+
+You can directly modify the dataframe by adding, renaming, or removing columns.
+
+```python
+# Create a new column
+cap_adata.obs['new_col'] = [value1, value2, value3]
+
+# Rename a column
+cap_adata.rename_column('old_col_name', 'new_col_name')
+
+# Remove a column
+cap_adata.remove_column('col_to_remove')
+```
+
+After modifications, you can overwrite the changes back to the AnnData file. If a value doesn't exist, it will be created.
+
+```python
+# overwrite all values which were read
+cap_adata.overwrite()
+
+# overwrite choosen fields
+cap_adata.overwrite(['obs', 'var'])
+```
+
+The full list of supported fields: `X`, `raw.X`, `obs`, `var`, `raw.var`, `obsm`, `uns`.
+
+#### 3. How to Read Few Columns but Overwrite One in a Dataframe
+
+The only way yet to do that is to drop all columns from in-memory dataframe (with `pandas.drop`!) before the call of `overwrite` method.
+
+```python
+# Read specific columns
+cap_adata.read_obs(columns=['cell_type', 'sample'])
+
+# Drop a column in-memory
+# DON'T USE remove_column here!
+cap_adata.obs.drop(columns='sample', inplace=True)
+
+# Overwrite changes
+cap_adata.overwrite(['obs'])
+```
+
+#### 4. How to work with X and raw.X
+
+The CapAnnData package won't read any field by default. However, the `X` and `raw.X` will be linked to the backed matrices automatically upon the first request to those fields.
+
+```python
+with h5py.File(path) as file:
+    # self.X is None here
+    cap_adata = CapAnnData(file)  
+
+    # will return the h5py.Dataset or CSRDataset
+    x = cap_adata.X  
+
+    # The same for raw.X
+    raw_x = cap_adata.raw.X 
+
+    # take whole matrix in memory
+    x = cap_adata.X[:] 
+```
+
+The CapAnnData supports the standard `numpy`/`h5py` sclising rules
+
+```python
+# slice rows
+s_ = np.s_[0:5]
+# slice columns
+s_ = np.s_[:, 0:5]
+# boolean mask + slicing
+mask = np.array([i < 5 for i in range(adata.shape[0])])
+s_ = np.s_[mask, :5]
+```
+
+#### 5. How to handle obsm embeddings matrixes
+
+By the default the CapAnnData will not read the embeddings matrix. The link to the h5py objects will be created upon the first call of the `.obsm` property. Alike the AnnData package the call like `cap_adata.obsm["X_tsne"]` will not return the in-memory matrix but will return the backed version instead. We can get the information about the name and shape of the embeddings without taking the whole matrixes in the memory!
+
+```python
+with h5py.File(path) as file:
+    # initialization
+    cap_adata = CapAnnData(file) 
+
+    # will return the list of strings
+    obsm_keys = cap_adata.obsm_keys()  
+
+    # return the shape of the matrix in backed mode
+    embeddings = obsm_keys[0]
+    shape = cap_adata.obsm[embeddings].shape  
+
+    # take the whole matrix in memory
+    matrix = cap_adata.obsm[embeddings][:]
+```
+
+#### 6. How to read and modify uns section
+
+The `CapAnnData` class will lazely link the uns section upon the first call but ***WILL NOT*** read it into memory. Instead, the dictionary of the pairs `{'key': "__NotLinkedObject"}` will be creted. It allow to get the list of keys before the actual read. To read the uns section in the memory the `.read_uns(keys)` method must be called.
+
+```python
+with h5py.File(path) as file:
+    # initialization
+    cap_adata = CapAnnData(file) 
+
+    # will return the keys() object
+    keys = cap_adata.uns.keys()  
+
+    # read in memory the first key only
+    cap_adata.read_uns([keys[0]])
+
+    # read the whole uns section into memory
+    cap_adata.read_uns()
+```
+
+Since the `.uns` section is in the memory (partially or completely) we can work with it as with the regular `dict()` python object. The main feature of the `CapAnnDataUns` class which inherited from `dict` is the tracking of the keys which must be removed from the `.h5ad` file upon overwrite. 
+
+```python
+# get the value
+v = cap_adata.uns["key1"]
+v = cap_adata.uns.get("key1")
+
+# modify values
+cap_adata.uns["key1"] = "new_value"
+
+# create new keys
+cap_adata.uns["new_key"] = "value"
+
+# remove keys
+cap_adata.uns.pop("key1")  # is recommended way
+del cap_adata.uns.pop("key2")
+cap_adata.uns.popitem()
+```
+
+To save `uns` changes the method `CapAnnData.overwrite()` must be called. 
+
+```python
+cap_adata.overwrite()  # all in-memory fields will be overwritten
+cap_adata.overwrite(["uns"])  # overwrite the uns secion only
+```

cap_anndata-0.1.0/cap_anndata/__init__.py

@@ -0,0 +1,6 @@
+from .backed_df import CapAnnDataDF
+from .backed_uns import CapAnnDataUns
+from .cap_anndata import CapAnnData
+
+
+__all__ = ["CapAnnData"]

cap_anndata-0.1.0/cap_anndata/backed_df.py

@@ -0,0 +1,40 @@
+import pandas as pd
+import numpy as np
+from typing import List
+import logging
+
+logger = logging.getLogger(__name__)
+
+
+class CapAnnDataDF(pd.DataFrame):
+    """
+    The class to expand the pandas DataFrame behaviour to support partial
+    reading and writing of AnnData obs and var (raw.var) fields.
+    The main feature of the class is handling <column-order> attribute
+    which must be a copy of h5py.Group attribute
+    """
+    _metadata = ['column_order']
+
+    def rename_column(self, old_name: str, new_name: str) -> None:
+        i = np.where(self.column_order == old_name)[0]
+        self.column_order[i] = new_name
+        self.rename(columns={old_name: new_name}, inplace=True)
+
+    def remove_column(self, col_name: str) -> None:
+        i = np.where(self.column_order == col_name)[0]
+        self.column_order = np.delete(self.column_order, i)
+        self.drop(columns=[col_name], inplace=True)
+
+    def __setitem__(self, key, value) -> None:
+        if key not in self.column_order:
+            self.column_order = np.append(self.column_order, key)
+        return super().__setitem__(key, value)
+
+    @classmethod
+    def from_df(cls, df: pd.DataFrame, column_order: List[str] = None):
+        if column_order is None:
+            column_order = df.columns.to_numpy()
+
+        new_inst = cls(df)
+        new_inst.column_order = column_order
+        return new_inst

cap_anndata-0.1.0/cap_anndata/backed_uns.py

@@ -0,0 +1,28 @@
+from typing import List, Any
+
+
+class CapAnnDataUns(dict):
+    __keys_to_remove: List[str] = []
+
+    def __delitem__(self, __key: Any) -> None:
+        self.__keys_to_remove.append(__key)
+        return super().__delitem__(__key)
+
+    def __setitem__(self, __key: Any, __value: Any) -> None:
+        if __key in self.__keys_to_remove:
+            self.__keys_to_remove.remove(__key)
+        return super().__setitem__(__key, __value)
+
+    @property
+    def keys_to_remove(self):
+        return self.__keys_to_remove
+
+    def pop(self, __key: Any, __default: Any = None) -> Any:
+        if __key in self:
+            self.__keys_to_remove.append(__key)
+        return super().pop(__key, __default)
+
+    def popitem(self) -> Any:
+        item = super().popitem()
+        self.__keys_to_remove.append(item[0])
+        return item

cap_anndata-0.1.0/cap_anndata/cap_anndata.py

@@ -0,0 +1,221 @@
+import logging
+import contextlib
+import anndata as ad
+import h5py
+from typing import List, Union, Dict, Tuple, Final
+from anndata._io.specs import read_elem, write_elem
+from dataclasses import dataclass
+
+from cap_anndata import CapAnnDataDF, CapAnnDataUns
+
+logger = logging.getLogger(__name__)
+
+X_NOTATION = Union[h5py.Dataset, ad.experimental.CSRDataset, ad.experimental.CSCDataset]
+OBSM_NOTATION = Dict[str, X_NOTATION]
+
+NotLinkedObject: Final = "__NotLinkedObject"
+
+
+@dataclass
+class RawLayer:
+    var: CapAnnDataDF = None
+    X: X_NOTATION = None
+
+    @property
+    def shape(self) -> Tuple[int, int]:
+        return self.X.shape if self.X is not None else None
+
+
+class CapAnnData:
+    def __init__(self, h5_file: h5py.File) -> None:
+        self._file: h5py.File = h5_file
+        self.obs: CapAnnDataDF = None
+        self.var: CapAnnDataDF = None
+        self._X: X_NOTATION = None
+        self._obsm: OBSM_NOTATION = None
+        self._uns: CapAnnDataUns = None
+        self._raw: RawLayer = None
+        self._shape: Tuple[int, int] = None
+
+    @property
+    def X(self) -> X_NOTATION:
+        if self._X is None:
+            self._link_x()
+        return self._X
+
+    @property
+    def obsm(self) -> OBSM_NOTATION:
+        if self._obsm is None:
+            self._link_obsm()
+        return self._obsm
+
+    @property
+    def raw(self) -> RawLayer:
+        if self._raw is None:
+            self._link_raw_x()
+        return self._raw
+
+    @property
+    def uns(self) -> CapAnnDataUns:
+        if self._uns is None:
+            self._uns = CapAnnDataUns({k: NotLinkedObject for k in self._file["uns"].keys()})
+        return self._uns
+
+    def read_obs(self, columns: List[str] = None) -> None:
+        self.obs = self._read_df(self._file["obs"], columns=columns)
+
+    def read_var(self, columns: List[str] = None, raw: bool = False) -> None:
+        if raw:
+            # Check if raw exists first
+            if "raw" not in self._file.keys():
+                logger.debug("Can't read raw.var since raw layer doesn't exist!")
+                return
+
+            if self._raw is None:
+                self._raw = RawLayer()
+                self._link_raw_x()
+
+            key = "raw/var"
+            self._raw.var = self._read_df(self._file[key], columns=columns)
+        else:
+            key = "var"
+            self.var = self._read_df(self._file[key], columns=columns)
+
+    def _read_df(self, h5_group: h5py.Group, columns: List[str]) -> CapAnnDataDF:
+        column_order = self._read_attr(h5_group, "column-order")
+
+        if columns is None:
+            # read whole df
+            df = CapAnnDataDF.from_df(read_elem(h5_group), column_order=column_order)
+        else:
+            cols_to_read = [c for c in columns if c in column_order]
+            df = CapAnnDataDF()
+            df.column_order = column_order
+
+            index_col = self._read_attr(h5_group, "_index")
+            df.index = read_elem(h5_group[index_col])
+
+            for col in cols_to_read:
+                df[col] = read_elem(h5_group[col])
+        return df
+
+    @staticmethod
+    def _read_attr(obj: Union[h5py.Group, h5py.Dataset], attr_name: str) -> any:
+        attrs = dict(obj.attrs)
+        if attr_name not in attrs.keys():
+            raise KeyError(f"The {attr_name} doesn't exist!")
+        return attrs[attr_name]
+
+    def overwrite(self, fields: List[str] = None) -> None:
+        field_to_entity = {
+            "obs": self.obs,
+            "var": self.var,
+            "raw.var": self.raw.var if self.raw is not None else None,
+            "uns": self.uns
+        }
+
+        if fields is None:
+            fields = list(field_to_entity.keys())
+        else:
+            for f in fields:
+                if f not in field_to_entity.keys():
+                    raise KeyError(
+                        f"The field {f} is not supported! The list of suported fields are equal to supported attributes of the CapAnnData class: obs, var, raw.var and uns.")
+
+        for key in ["obs", "var", "raw.var"]:
+            if key in fields:
+                entity: CapAnnDataDF = field_to_entity[key]
+                if entity is None:
+                    continue
+
+                key = key.replace(".", '/') if key == "raw.var" else key
+
+                for col in entity.columns:
+                    self._write_elem_lzf(f"{key}/{col}", entity[col].values)
+                self._file[key].attrs['column-order'] = entity.column_order
+
+        if "uns" in fields:
+            for key in self.uns.keys():
+                if self.uns[key] is not NotLinkedObject:
+                    dest = f"uns/{key}"
+                    self._write_elem_lzf(dest, self.uns[key])
+            for key in self.uns.keys_to_remove:
+                del self._file[f"uns/{key}"]
+
+    def read_uns(self, keys: List[str] = None) -> None:
+        if keys is None:
+            keys = list(self.uns.keys())
+
+        for key in keys:
+            existing_keys = self.uns.keys()
+            if key in existing_keys:
+                sourse = self._file[f"uns/{key}"]
+                self.uns[key] = read_elem(sourse)
+
+    @property
+    def shape(self) -> tuple[int, int]:
+        return self.X.shape
+
+    def _link_x(self) -> None:
+        x = self._file["X"]
+        if isinstance(x, h5py.Dataset):
+            # dense X
+            self._X = x
+        else:
+            # sparse dataset
+            self._X = ad.experimental.sparse_dataset(x)
+
+    def _link_raw_x(self) -> None:
+        if "raw" in self._file.keys():
+            if self._raw is None:
+                self._raw = RawLayer()
+
+            raw_x = self._file["raw/X"]
+            if isinstance(raw_x, h5py.Dataset):
+                # dense X
+                self._raw.X = raw_x
+            else:
+                # sparse dataset
+                self._raw.X = ad.experimental.sparse_dataset(raw_x)
+
+    def _link_obsm(self) -> None:
+        self._obsm = {}
+        if "obsm" in self._file.keys():
+            obsm_group = self._file["obsm"]
+            for entity_name in obsm_group.keys():
+                entity = obsm_group[entity_name]
+                if isinstance(entity, h5py.Dataset):
+                    # dense array
+                    self._obsm[entity_name] = entity
+                else:
+                    # sparse array
+                    self._obsm[entity_name] = ad.experimental.sparse_dataset(entity)
+        logger.debug(f"obsm={self._obsm}")
+
+    def obsm_keys(self) -> List[str]:
+        return list(self.obsm.keys())
+
+    def _write_elem_lzf(self, dest_key: str, elem: any) -> None:
+        write_elem(self._file, dest_key, elem, dataset_kwargs={"compression": "lzf"})
+
+    @staticmethod
+    @contextlib.contextmanager
+    def read_anndata_file(file_path, backed='r'):
+        """The method to read anndata file using original AnnData package"""
+        logger.debug(f"Read file {file_path} in backed mode = {backed}...")
+
+        adata = None
+        try:
+            adata = ad.read_h5ad(file_path, backed=backed)
+            logger.debug(f"Successfully read anndata file path {file_path}")
+            yield adata
+
+        except Exception as error:
+            logger.error(f"Error during read anndata file at path: {file_path}, error = {error}!")
+            raise error
+
+        finally:
+            if adata is not None:
+                if adata.isbacked:
+                    adata.file.close()
+                logger.debug("AnnData closed!")

cap_anndata-0.1.0/cap_anndata.egg-info/PKG-INFO

@@ -0,0 +1,199 @@
+Metadata-Version: 2.1
+Name: cap_anndata
+Version: 0.1.0
+Summary: Partial read of AnnData files for low-memory operations with large datasets.
+Home-page: https://github.com/cellannotation/cap-anndata
+Author: R. Mukhin, A. Isaev
+Author-email: roman@ebookapplications.com
+Project-URL: Bug Tracker, https://github.com/cellannotation/cap-anndata/issues
+Classifier: Programming Language :: Python :: 3.9
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.26.3
+Requires-Dist: pandas>=2.2.0
+Requires-Dist: anndata>=0.10.5
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0.0; extra == "dev"
+
+# CAP-AnnData: Enhanced Partial I/O for AnnData Files
+
+## Overview
+CAP-AnnData enriches the AnnData ecosystem by offering tailored functionalities for partial reading and writing of AnnData files. This enhancement allows for selective manipulation of sections such as `obs`, `var`, `X`, `raw.X`, `obsm`, and `uns` without the need for loading entire datasets into memory. Leveraging AnnData's native methods, CAP-AnnData aims to maintain backward compatibility while improving efficiency, especially useful for large-scale single-cell genomics data.
+
+## Getting Started
+
+### Running Tests
+Ensure the integrity and reliability of CAP-AnnData on your system by running the unit tests in `test/unit_test.py`.
+
+Make sure Python 3.9 or newer is used, along with all requirements specified in requirements.txt
+
+## How-TO:
+
+#### 1. Read AnnData File Dataframes
+
+##### Basic Reading
+By default, `CapAnnData` does not automatically read any data. To begin working with dataframes, you need to explicitly read the data from the AnnData file. You can read the entire dataframe or select specific columns. For partial reading, provide a list of column names.
+
+```python
+import h5py
+from cap_anndata import CapAnnData
+
+file_path = "your_data.h5ad"
+with h5py.File(file_path, 'r') as file:
+    cap_adata = CapAnnData(file)
+
+    # Read all columns of 'obs'
+    cap_adata.read_obs()
+
+    # Read specific columns of 'var'
+    cap_adata.read_var(columns=['gene_expression', 'dispersion'])
+
+    # Read all columns of raw.var
+    cap_adata.read_var(raw=True)
+```
+
+##### Non-existing columns
+
+If a column doesn't exist in the file, no error will be raised but the column will be missing in the resulting Dataframe. So, the list of columns saying more like "try to read this columns from the file". It is needed because we there is no way yet to check if the column exists before the read. 
+
+#### 2. Modify the AnnData File Dataframes In-Place
+
+You can directly modify the dataframe by adding, renaming, or removing columns.
+
+```python
+# Create a new column
+cap_adata.obs['new_col'] = [value1, value2, value3]
+
+# Rename a column
+cap_adata.rename_column('old_col_name', 'new_col_name')
+
+# Remove a column
+cap_adata.remove_column('col_to_remove')
+```
+
+After modifications, you can overwrite the changes back to the AnnData file. If a value doesn't exist, it will be created.
+
+```python
+# overwrite all values which were read
+cap_adata.overwrite()
+
+# overwrite choosen fields
+cap_adata.overwrite(['obs', 'var'])
+```
+
+The full list of supported fields: `X`, `raw.X`, `obs`, `var`, `raw.var`, `obsm`, `uns`.
+
+#### 3. How to Read Few Columns but Overwrite One in a Dataframe
+
+The only way yet to do that is to drop all columns from in-memory dataframe (with `pandas.drop`!) before the call of `overwrite` method.
+
+```python
+# Read specific columns
+cap_adata.read_obs(columns=['cell_type', 'sample'])
+
+# Drop a column in-memory
+# DON'T USE remove_column here!
+cap_adata.obs.drop(columns='sample', inplace=True)
+
+# Overwrite changes
+cap_adata.overwrite(['obs'])
+```
+
+#### 4. How to work with X and raw.X
+
+The CapAnnData package won't read any field by default. However, the `X` and `raw.X` will be linked to the backed matrices automatically upon the first request to those fields.
+
+```python
+with h5py.File(path) as file:
+    # self.X is None here
+    cap_adata = CapAnnData(file)  
+
+    # will return the h5py.Dataset or CSRDataset
+    x = cap_adata.X  
+
+    # The same for raw.X
+    raw_x = cap_adata.raw.X 
+
+    # take whole matrix in memory
+    x = cap_adata.X[:] 
+```
+
+The CapAnnData supports the standard `numpy`/`h5py` sclising rules
+
+```python
+# slice rows
+s_ = np.s_[0:5]
+# slice columns
+s_ = np.s_[:, 0:5]
+# boolean mask + slicing
+mask = np.array([i < 5 for i in range(adata.shape[0])])
+s_ = np.s_[mask, :5]
+```
+
+#### 5. How to handle obsm embeddings matrixes
+
+By the default the CapAnnData will not read the embeddings matrix. The link to the h5py objects will be created upon the first call of the `.obsm` property. Alike the AnnData package the call like `cap_adata.obsm["X_tsne"]` will not return the in-memory matrix but will return the backed version instead. We can get the information about the name and shape of the embeddings without taking the whole matrixes in the memory!
+
+```python
+with h5py.File(path) as file:
+    # initialization
+    cap_adata = CapAnnData(file) 
+
+    # will return the list of strings
+    obsm_keys = cap_adata.obsm_keys()  
+
+    # return the shape of the matrix in backed mode
+    embeddings = obsm_keys[0]
+    shape = cap_adata.obsm[embeddings].shape  
+
+    # take the whole matrix in memory
+    matrix = cap_adata.obsm[embeddings][:]
+```
+
+#### 6. How to read and modify uns section
+
+The `CapAnnData` class will lazely link the uns section upon the first call but ***WILL NOT*** read it into memory. Instead, the dictionary of the pairs `{'key': "__NotLinkedObject"}` will be creted. It allow to get the list of keys before the actual read. To read the uns section in the memory the `.read_uns(keys)` method must be called.
+
+```python
+with h5py.File(path) as file:
+    # initialization
+    cap_adata = CapAnnData(file) 
+
+    # will return the keys() object
+    keys = cap_adata.uns.keys()  
+
+    # read in memory the first key only
+    cap_adata.read_uns([keys[0]])
+
+    # read the whole uns section into memory
+    cap_adata.read_uns()
+```
+
+Since the `.uns` section is in the memory (partially or completely) we can work with it as with the regular `dict()` python object. The main feature of the `CapAnnDataUns` class which inherited from `dict` is the tracking of the keys which must be removed from the `.h5ad` file upon overwrite. 
+
+```python
+# get the value
+v = cap_adata.uns["key1"]
+v = cap_adata.uns.get("key1")
+
+# modify values
+cap_adata.uns["key1"] = "new_value"
+
+# create new keys
+cap_adata.uns["new_key"] = "value"
+
+# remove keys
+cap_adata.uns.pop("key1")  # is recommended way
+del cap_adata.uns.pop("key2")
+cap_adata.uns.popitem()
+```
+
+To save `uns` changes the method `CapAnnData.overwrite()` must be called. 
+
+```python
+cap_adata.overwrite()  # all in-memory fields will be overwritten
+cap_adata.overwrite(["uns"])  # overwrite the uns secion only
+```

cap_anndata-0.1.0/cap_anndata.egg-info/SOURCES.txt

@@ -0,0 +1,15 @@
+LICENSE
+README.md
+setup.py
+cap_anndata/__init__.py
+cap_anndata/backed_df.py
+cap_anndata/backed_uns.py
+cap_anndata/cap_anndata.py
+cap_anndata.egg-info/PKG-INFO
+cap_anndata.egg-info/SOURCES.txt
+cap_anndata.egg-info/dependency_links.txt
+cap_anndata.egg-info/requires.txt
+cap_anndata.egg-info/top_level.txt
+test/test_backed_df.py
+test/test_backed_uns.py
+test/test_cap_anndata.py

cap_anndata-0.1.0/cap_anndata.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

cap_anndata-0.1.0/cap_anndata.egg-info/requires.txt

@@ -0,0 +1,6 @@
+numpy>=1.26.3
+pandas>=2.2.0
+anndata>=0.10.5
+
+[dev]
+pytest>=8.0.0

cap_anndata-0.1.0/cap_anndata.egg-info/top_level.txt

@@ -0,0 +1 @@
+cap_anndata

cap_anndata-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

cap_anndata-0.1.0/setup.py

@@ -0,0 +1,28 @@
+from setuptools import setup, find_packages
+
+setup(
+    name='cap_anndata',
+    version='0.1.0',
+    author='R. Mukhin, A. Isaev',
+    author_email='roman@ebookapplications.com',
+    packages=find_packages(exclude=["test"]),
+    description='Partial read of AnnData files for low-memory operations with large datasets.',
+    long_description=open('README.md').read(),
+    long_description_content_type='text/markdown',
+    url='https://github.com/cellannotation/cap-anndata',
+    project_urls={
+        "Bug Tracker": "https://github.com/cellannotation/cap-anndata/issues"
+    },
+    classifiers=[
+        "Programming Language :: Python :: 3.9",
+        "License :: OSI Approved :: BSD License",
+        "Operating System :: OS Independent",
+    ],
+    python_requires='>=3.9',
+    install_requires=[
+        "numpy>=1.26.3",
+        "pandas>=2.2.0",
+        "anndata>=0.10.5"
+    ],
+    extras_require={"dev": ["pytest>=8.0.0"]}
+)

cap_anndata-0.1.0/test/test_backed_df.py

@@ -0,0 +1,58 @@
+import pandas as pd
+import numpy as np
+from cap_anndata import CapAnnDataDF
+
+
+
+def test_from_df():
+    data = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})
+    cap_ann_data_df = CapAnnDataDF.from_df(data)
+
+    assert np.allclose(data.values, cap_ann_data_df.values)
+    assert all(data.columns == cap_ann_data_df.column_order)
+
+
+def test_create_column():
+    data = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})
+    cap_ann_data_df = CapAnnDataDF.from_df(data, column_order=["A", "B", "D"])
+    cap_ann_data_df["C"] = [7, 8, 9]
+
+    assert 'C' in cap_ann_data_df.columns
+    assert 'C' in cap_ann_data_df.column_order
+
+
+def test_rename_column():
+    data = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})
+    cap_ann_data_df = CapAnnDataDF.from_df(data)
+    cap_ann_data_df.rename_column('A', 'A_renamed')
+
+    assert 'A_renamed' in cap_ann_data_df.columns
+    assert 'A' not in cap_ann_data_df.columns
+    assert 'A_renamed' in cap_ann_data_df.column_order
+
+
+def test_remove_column():
+    data = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})
+    cap_ann_data_df = CapAnnDataDF.from_df(data)
+    cap_ann_data_df.remove_column('B')
+
+    assert 'B' not in cap_ann_data_df.columns
+    assert 'B' not in cap_ann_data_df.column_order
+
+
+def test_from_df_class_method():
+    data = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})
+    new_df = CapAnnDataDF.from_df(data, ['B', 'A'])
+
+    assert list(new_df.column_order) == ['B', 'A']
+
+
+def test_column_order_integrity():
+    data = pd.DataFrame({'A': [1, 2, 3], 'B': [4, 5, 6]})
+    cap_ann_data_df = CapAnnDataDF.from_df(data)
+    cap_ann_data_df["C"] = [7, 8, 9]
+    cap_ann_data_df.rename_column('A', 'A_renamed')
+    cap_ann_data_df.remove_column('B')
+
+    expected_order = ['A_renamed', 'C']
+    assert list(cap_ann_data_df.column_order) == expected_order

cap_anndata-0.1.0/test/test_backed_uns.py

@@ -0,0 +1,36 @@
+from cap_anndata import CapAnnDataUns
+
+
+def test_basic_init():
+    d = {'key1': 'value1', 'key2': {'value2': 'sub_value2'}}
+    cap_d = CapAnnDataUns(d)
+    assert cap_d == d
+    assert cap_d.keys_to_remove == []
+    assert cap_d.get("key1") == "value1"
+    assert cap_d.get("key_N") == None
+
+
+def test_pop():
+    cap_d = CapAnnDataUns()
+    cap_d["key1"] = "value1"
+    cap_d["key2"] = "value2"
+
+    cap_d.pop("key1")
+    
+    assert len(cap_d.keys()) == 1
+    assert cap_d.keys_to_remove == ["key1"]
+
+    cap_d["key1"] = "new_value"
+
+    assert len(cap_d.keys()) == 2
+    assert cap_d.keys_to_remove == []
+
+    cap_d.popitem()
+    assert len(cap_d.keys()) == 1
+    assert cap_d.keys_to_remove == ["key1"]
+
+    cap_d["key1"] = {'sk1': 'v1', "sk2": 'v2'}
+    cap_d["key1"].pop('sk2')
+
+    assert len(cap_d.keys()) == 2
+    assert cap_d.keys_to_remove == []

cap_anndata-0.1.0/test/test_cap_anndata.py

@@ -0,0 +1,291 @@
+from cap_anndata import CapAnnData
+import anndata as ad
+import numpy as np
+import tempfile
+import os
+import h5py
+import pandas as pd
+import scipy.sparse as sp
+import pytest
+
+
+def get_base_anndata(n_rows: int = 10, n_genes: int = 10, sparse=False) -> ad.AnnData:
+    x = np.eye(n_rows, n_genes).astype(np.float32)
+    if sparse:
+        x = sp.csr_matrix(x, dtype=np.float32)
+    adata = ad.AnnData(X=x)
+    return adata
+
+
+def get_filled_anndata(n_rows: int = 10, n_genes: int = 10, sparse=False) -> ad.AnnData:
+    adata = get_base_anndata(n_rows, n_genes, sparse)
+
+    adata.obs["cell_type"] = [f"cell_{i%3}" for i in range(adata.shape[0])]
+    adata.obs["number"] = [i / 10 for i in range(adata.shape[0])]
+    adata.obs.index = [f"obs_{i}" for i in range(adata.shape[0])]
+
+    adata.var.index = [f"gene_{i}" for i in range(adata.shape[1])]
+    adata.var["filtered"] = [i > 4 for i in range(adata.shape[1])]
+    adata.var["gene_names"] = [f"gene_name_{i}" for i in range(adata.shape[1])]
+    adata.var["dispersion"] = [i / 100 for i in range(adata.shape[1])]
+
+    adata.raw = adata
+    return adata
+
+    
+def test_read_anndata_file():
+    adata = get_base_anndata()
+    temp_folder = tempfile.mkdtemp()
+    file_path = os.path.join(temp_folder, "test_read_anndata_file.h5ad")
+    adata.write_h5ad(file_path)
+    del adata
+
+    with CapAnnData.read_anndata_file(file_path=file_path) as adata:
+        assert adata is not None, "AnnData file must be valid!"
+
+    os.remove(file_path)
+
+
+def test_read_shape():
+    n_rows = 10
+    n_genes = 20
+    adata = get_base_anndata(n_rows, n_genes)
+    temp_folder = tempfile.mkdtemp()
+    file_path = os.path.join(temp_folder, "test_read_shape.h5ad")
+    adata.write_h5ad(file_path)
+
+    with h5py.File(file_path) as file:
+        cap_adata = CapAnnData(file)
+        shape = cap_adata.shape
+    
+    os.remove(file_path)
+    assert shape[0] == n_rows
+    assert shape[1] == n_genes
+
+
+def test_read_df():
+    adata = get_filled_anndata()
+    temp_folder = tempfile.mkdtemp()
+    file_path = os.path.join(temp_folder, "test_read_obs.h5ad")
+
+    adata.write_h5ad(file_path)
+
+    with h5py.File(file_path, 'r') as file:
+        cap_adata = CapAnnData(file)
+        cap_adata.read_obs()
+        cap_adata.read_var()
+        cap_adata.read_var(raw=True)
+
+    os.remove(file_path)
+    pd.testing.assert_frame_equal(adata.obs, cap_adata.obs, check_frame_type=False)
+    pd.testing.assert_frame_equal(adata.var, cap_adata.var, check_frame_type=False)
+    pd.testing.assert_frame_equal(adata.raw.var, cap_adata.raw.var, check_frame_type=False)
+
+
+def test_partial_read():
+    adata = get_filled_anndata()
+    temp_folder = tempfile.mkdtemp()
+    file_path = os.path.join(temp_folder, "test_partial_read.h5ad")
+    adata.write_h5ad(file_path)
+
+    with h5py.File(file_path, 'r') as file:
+        cap_adata = CapAnnData(file)
+        cap_adata.read_obs(columns=['cell_type'])
+        cap_adata.read_obs(columns=['cell_type'])
+        cap_adata.read_var(columns=['dispersion'])
+        cap_adata.read_var(columns=['dispersion'], raw=True)
+    
+    os.remove(file_path)
+
+    assert len(adata.obs.columns) == len(cap_adata.obs.column_order)
+    assert len(adata.var.columns) == len(cap_adata.var.column_order)
+    assert len(adata.raw.var.columns) == len(cap_adata.raw.var.column_order)
+
+    assert len(cap_adata.obs.columns) == 1
+    assert len(cap_adata.var.columns) == 1
+    assert len(cap_adata.raw.var.columns) == 1
+
+    pd.testing.assert_index_equal(adata.obs.index, cap_adata.obs.index)
+    pd.testing.assert_index_equal(adata.var.index, cap_adata.var.index)
+    pd.testing.assert_index_equal(adata.raw.var.index, cap_adata.raw.var.index)
+
+
+def test_overwrite_df():
+    adata = get_filled_anndata()
+    temp_folder = tempfile.mkdtemp()
+    file_path = os.path.join(temp_folder, "test_overwrite_df.h5ad")
+    adata.write_h5ad(file_path)
+
+    with h5py.File(file_path, 'r+') as file:
+        cap_adata = CapAnnData(file)
+        cap_adata.read_obs(columns=["cell_type"])
+        cap_adata.obs["cell_type"] = [f"new_cell_type_{i%2}" for i in range(cap_adata.shape[0])]
+        cap_adata.obs["const_str"] = "some string"
+        ref_obs = cap_adata.obs.copy()
+
+        # Modify 'var'
+        cap_adata.read_var()
+        cap_adata.var["gene_names"] = [f"new_gene_{i}" for i in range(cap_adata.shape[1])]
+        cap_adata.var["extra_info"] = np.random.rand(cap_adata.shape[1])
+        ref_var = cap_adata.var.copy()
+
+        # Modify 'raw.var', assuming 'raw' is also a CapAnnData
+        cap_adata.read_var(raw=True)
+        cap_adata.raw.var["gene_names"] = [f"raw_new_gene_{i}" for i in range(cap_adata.raw.shape[1])]
+        cap_adata.raw.var["extra_info"] = np.random.rand(cap_adata.shape[1])
+        ref_raw_var = cap_adata.raw.var.copy()
+
+        cap_adata.overwrite(['obs', 'var', 'raw.var'])
+
+    adata = ad.read_h5ad(file_path)
+    os.remove(file_path)
+    
+    # Assert changes in 'obs'
+    assert all([c in adata.obs.columns for c in ref_obs.columns])
+    pd.testing.assert_frame_equal(ref_obs, adata.obs[ref_obs.columns.to_list()], check_frame_type=False)
+
+    # Assert changes in 'var'
+    assert all([c in adata.var.columns for c in ref_var.columns])
+    pd.testing.assert_frame_equal(ref_var, adata.var[ref_var.columns.to_list()], check_frame_type=False)
+
+    # Assert changes in 'raw.var'
+    assert all([c in adata.raw.var.columns for c in ref_raw_var.columns])
+    pd.testing.assert_frame_equal(ref_raw_var, adata.raw.var[ref_raw_var.columns.to_list()], check_frame_type=False)
+
+
+@pytest.mark.parametrize("sparse", [False, True])
+@pytest.mark.parametrize("vertical_slice", [None, False, True, "mask"])
+def test_link_x(sparse, vertical_slice):
+    adata = get_filled_anndata(sparse=sparse)
+    temp_folder = tempfile.mkdtemp()
+    file_path = os.path.join(temp_folder, "test_link_x.h5ad")
+    adata.write_h5ad(file_path)
+
+    if vertical_slice is None:
+        s_ = np.s_[:]
+    elif vertical_slice == "mask":
+        mask = np.array([i < 5 for i in range(adata.shape[0])])
+        s_ = np.s_[mask, :5]
+    else:
+        # slice over var or obs
+        s_ = np.s_[:, 0:5] if vertical_slice else np.s_[0:5, :]
+
+    with h5py.File(file_path, 'r') as file:
+        cap_adata = CapAnnData(file)
+        x = cap_adata.X[s_]
+        raw_x = cap_adata.raw.X[s_]
+    
+    os.remove(file_path)
+    if sparse:
+        assert np.allclose(adata.X.A[s_], x.A)
+        assert np.allclose(adata.raw.X.A[s_], raw_x.A)
+    else:
+        assert np.allclose(adata.X[s_], x)
+        assert np.allclose(adata.raw.X[s_], raw_x)
+
+
+@pytest.mark.parametrize("sparse", [False, True])
+def test_shape(sparse):
+    n_rows = 15
+    n_genes = 25
+
+    adata = get_filled_anndata(n_rows, n_genes, sparse)
+    temp_folder = tempfile.mkdtemp()
+    file_path = os.path.join(temp_folder, "test_shape.h5ad")
+    adata.write_h5ad(file_path)
+
+    with h5py.File(file_path) as file:
+        cap_adata = CapAnnData(file)
+        shape = cap_adata.shape
+        shape_raw = cap_adata.raw.shape
+
+    os.remove(file_path)
+    for sh in [shape, shape_raw]:
+        assert sh == (n_rows, n_genes)
+
+
+def test_read_obsm():
+    adata = get_filled_anndata()
+    obsm_names = [f"X_test{i}" for i in range(2)]
+
+    for emb in obsm_names:
+        adata.obsm[emb] = np.random.random(size=(adata.shape[0], 2))
+
+    temp_folder = tempfile.mkdtemp()
+    file_path = os.path.join(temp_folder, "test_read_obsm.h5ad")
+    adata.write_h5ad(file_path)
+
+    with h5py.File(file_path, 'r') as f:
+        cap_adata = CapAnnData(f)
+        
+        ss = []
+        for emb in obsm_names:
+            assert emb in cap_adata.obsm_keys()
+            assert cap_adata.obsm[emb].shape == adata.obsm[emb].shape
+        
+        x_1 = cap_adata.obsm[obsm_names[0]][:]
+        x_2 = cap_adata.obsm[obsm_names[1]][:]
+
+    os.remove(file_path)
+    assert np.allclose(adata.obsm[obsm_names[0]], x_1)
+    assert np.allclose(adata.obsm[obsm_names[1]], x_2)
+
+
+def test_read_uns():
+    adata = get_base_anndata()
+    key1, key2 = "key1", "key2"
+    keys = (key1, key2)
+    
+    adata.uns = {k: {k: k} for k in keys}
+    temp_folder = tempfile.mkdtemp()
+    file_path = os.path.join(temp_folder, "test_read_uns.h5ad")
+    adata.write_h5ad(file_path)
+
+    with h5py.File(file_path, 'r') as f:
+        cap_adata = CapAnnData(f)
+
+        for k in keys:
+            assert k in cap_adata.uns
+        
+        cap_adata.read_uns(keys=[key1])
+
+        assert cap_adata.uns[key1] == adata.uns[key1]  # connected
+        assert cap_adata.uns[key2] != adata.uns[key2]  # not connected
+
+    os.remove(file_path)
+
+
+def test_modify_uns():
+    adata = get_base_anndata()
+    adata.uns = {
+        "field_to_ingore": list(range(100)),
+        "field_to_rename": "value",
+        "field_to_expand": {"key1": {}},
+        "field_to_modify": {"a": "b"}
+    }
+    new_name = "renamed_field"
+    d_to_exp = {"sub_key1": "v1", "sub_key2": "v2"}
+    v_to_mod = "value"
+
+    temp_folder = tempfile.mkdtemp()
+    file_path = os.path.join(temp_folder, "test_modify_uns.h5ad")
+    adata.write_h5ad(file_path)
+
+    with h5py.File(file_path, 'r+') as f:
+        cap_adata = CapAnnData(f)
+
+        cap_adata.read_uns(keys=["field_to_rename", "field_to_expand", "field_to_modify"])
+
+        cap_adata.uns[new_name] = cap_adata.uns.pop("field_to_rename")
+        cap_adata.uns["field_to_expand"]["key1"] = d_to_exp
+        cap_adata.uns["field_to_modify"] = v_to_mod
+
+        cap_adata.overwrite(['uns'])
+    
+    adata = ad.read_h5ad(file_path)
+
+    assert adata.uns is not None
+    assert len(adata.uns.keys()) == 4
+    assert new_name in adata.uns.keys()
+    assert adata.uns['field_to_expand']["key1"] == d_to_exp
+    assert adata.uns['field_to_modify'] == v_to_mod