lamindb 0.75.0__tar.gz → 0.75.1__tar.gz

{lamindb-0.75.0 → lamindb-0.75.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: lamindb
-Version: 0.75.0
+Version: 0.75.1
 Summary: A data framework for biology.
 Author-email: Lamin Labs <open-source@lamin.ai>
 Requires-Python: >=3.8
@@ -9,8 +9,8 @@ Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
-Requires-Dist: lnschema_core==0.72.1
-Requires-Dist: lamindb_setup==0.76.2
+Requires-Dist: lnschema_core==0.72.2
+Requires-Dist: lamindb_setup==0.76.3
 Requires-Dist: lamin_utils==0.13.2
 Requires-Dist: lamin_cli==0.16.0
 Requires-Dist: rapidfuzz
@@ -24,7 +24,7 @@ Requires-Dist: pandas
 Requires-Dist: graphviz
 Requires-Dist: psycopg2-binary
 Requires-Dist: lamindb_setup[aws] ; extra == "aws"
-Requires-Dist: bionty==0.47.1 ; extra == "bionty"
+Requires-Dist: bionty==0.48.0 ; extra == "bionty"
 Requires-Dist: pre-commit ; extra == "dev"
 Requires-Dist: nox ; extra == "dev"
 Requires-Dist: laminci>=0.3 ; extra == "dev"

{lamindb-0.75.0 → lamindb-0.75.1}/docs/api.md

@@ -20,7 +20,7 @@ lamindb
 
 ```{toctree}
 :maxdepth: 1
-:caption: Plug-ins
+:caption: Plugins
 :hidden:
 
 bionty

{lamindb-0.75.0 → lamindb-0.75.1}/docs/bio-registries.ipynb

@@ -20,7 +20,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "With plug-in {mod}`bionty`, it's easy to import records from public biological ontologies."
+    "This guide shows how to manage metadata for basic biological entities based on plugin {mod}`bionty`."
    ]
   },
   {
@@ -541,7 +541,7 @@
    "source": [
     "Several registries are organism-aware (has a `.organism` field), for instance, {class}`~bionty.Gene`.\n",
     "\n",
-    "In this case, API calls that interact with multi-organism registries requires a `organism` argument (name or {class}`~bionty.CellType`) when there's ambiguity."
+    "In this case, API calls that interact with multi-organism registries require an `organism` argument when there's ambiguity."
    ]
   },
   {
@@ -584,7 +584,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "And when working with the same organism throughout your analysis/workflow, you can omit the `organism` argument by configuring it globally:"
+    "When working with the same organism throughout your analysis/workflow, you can omit the `organism` argument by configuring it globally:"
    ]
   },
   {
@@ -757,7 +757,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.13"
+   "version": "3.11.7"
   },
   "nbproject": {
    "id": "s7F7PezA1HdQ",

{lamindb-0.75.0 → lamindb-0.75.1}/docs/curate-df.ipynb

@@ -75,12 +75,15 @@
    },
    "outputs": [],
    "source": [
-    "df = pd.DataFrame({\n",
-    "    \"temperature\": [37.2, 36.3, 38.2],\n",
-    "    \"cell_type\": [\"cerebral pyramidal neuron\", \"astrocyte\", \"oligodendrocyte\"],\n",
-    "    \"assay_ontology_id\": [\"EFO:0008913\", \"EFO:0008913\", \"EFO:0008913\"],\n",
-    "    \"donor\": [\"D0001\", \"D0002\", \"DOOO3\"],\n",
-    "})\n",
+    "df = pd.DataFrame(\n",
+    "    {\n",
+    "        \"temperature\": [37.2, 36.3, 38.2],\n",
+    "        \"cell_type\": [\"cerebral pyramidal neuron\", \"astrocyte\", \"oligodendrocyte\"],\n",
+    "        \"assay_ontology_id\": [\"EFO:0008913\", \"EFO:0008913\", \"EFO:0008913\"],\n",
+    "        \"donor\": [\"D0001\", \"D0002\", \"DOOO3\"]\n",
+    "    },\n",
+    "    index = [\"obs1\", \"obs2\", \"obs3\"]\n",
+    ")\n",
     "df"
    ]
   },
@@ -109,8 +112,24 @@
     "    \"cell_type\": bt.CellType.name,\n",
     "    \"assay_ontology_id\": bt.ExperimentalFactor.ontology_id,\n",
     "    \"donor\": ln.ULabel.name,\n",
-    "}\n",
-    "\n",
+    "}"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6c293ddf",
+   "metadata": {},
+   "source": [
+    "Now, initialize a Curate object by passing data and its validation criteria:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "78424b30",
+   "metadata": {},
+   "outputs": [],
+   "source": [
     "# create an Curate object to guide validation and annotation\n",
     "# this object will use our DataFrame and the defined categorical criteria\n",
     "curate = ln.Curate.from_df(df, categoricals=categoricals)"
@@ -150,17 +169,6 @@
     "Because our current database instance is empty, here, we'll add values to the registries defined in the validation criteria."
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "65bb5e39",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# this adds assays that were validated (via a public ontology)\n",
-    "curate.add_validated_from(\"assay_ontology_id\")"
-   ]
-  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -172,8 +180,8 @@
    },
    "outputs": [],
    "source": [
-    "# this adds cell types that were _not_ validated\n",
-    "curate.add_new_from(\"cell_type\")"
+    "# this adds cell types that were validated (via a public ontology)\n",
+    "curate.add_validated_from(\"cell_type\")"
    ]
   },
   {
@@ -220,10 +228,25 @@
    "source": [
     "# curate the cell type\n",
     "df.cell_type = df.cell_type.replace({\"cerebral pyramidal neuron\": cell_types.cerebral_cortex_pyramidal_neuron.name})\n",
-    "# register validated cell types\n",
+    "# now register curated and validated cell types\n",
     "curate.add_validated_from(df.cell_type.name)"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ede057e8",
+   "metadata": {
+    "tags": [
+     "hide-output"
+    ]
+   },
+   "outputs": [],
+   "source": [
+    "# this adds assays that were validated (via a public ontology)\n",
+    "curate.add_validated_from(\"assay_ontology_id\")"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -235,7 +258,7 @@
    },
    "outputs": [],
    "source": [
-    "# register non-validated donors\n",
+    "# this adds donors that were _not_ validated\n",
     "curate.add_new_from(df.donor.name)"
    ]
   },
@@ -262,7 +285,7 @@
    "source": [
     "## Validate an AnnData object\n",
     "\n",
-    "Here we addtionally specify which `var_fields` to validate against."
+    "Here we addtionally specify which `var_index` to validate against."
    ]
   },
   {
@@ -276,9 +299,17 @@
    },
    "outputs": [],
    "source": [
-    "df.index = [\"obs1\", \"obs2\", \"obs3\"]\n",
-    "\n",
-    "X = pd.DataFrame({\"TCF7\": [1, 2, 3], \"PDCD1\": [4, 5, 6], \"CD3E\": [7, 8, 9], \"CD4\": [10, 11, 12], \"CD8A\": [13, 14, 15]}, index=[\"obs1\", \"obs2\", \"obs3\"])\n",
+    "X = pd.DataFrame(\n",
+    "    {\n",
+    "        \"ENSG00000081059\": [1, 2, 3], \n",
+    "        \"ENSG00000276977\": [4, 5, 6], \n",
+    "        \"ENSG00000198851\": [7, 8, 9], \n",
+    "        \"ENSG00000010610\": [10, 11, 12], \n",
+    "        \"ENSG00000153563\": [13, 14, 15],\n",
+    "        \"corrupted\": [16, 17, 18]\n",
+    "    }, \n",
+    "    index=df.index\n",
+    ")\n",
     "\n",
     "adata = ad.AnnData(X=X, obs=df)\n",
     "adata"
@@ -297,7 +328,7 @@
    "source": [
     "curate = ln.Curate.from_anndata(\n",
     "    adata, \n",
-    "    var_index=bt.Gene.symbol,\n",
+    "    var_index=bt.Gene.ensembl_gene_id,  # validate var.index against Gene.ensembl_gene_id\n",
     "    categoricals=categoricals, \n",
     "    organism=\"human\",\n",
     ")"
@@ -317,24 +348,62 @@
     "curate.validate()"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "a8a7a653",
+   "metadata": {},
+   "source": [
+    "## Curate data object to pass validation"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "614545ea",
+   "metadata": {},
+   "source": [
+    "Non-validated terms can be accessed via:"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "7877c058",
-   "metadata": {
-    "tags": [
-     "hide-output"
-    ]
-   },
+   "id": "4e2fd290",
+   "metadata": {},
    "outputs": [],
    "source": [
-    "curate.add_validated_from(\"all\")"
+    "curate.non_validated"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b2ec4aeb",
+   "metadata": {},
+   "source": [
+    "Subset anndata object to validated genes only:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "3c47e4b7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "adata_validated = adata[:, ~adata.var.index.isin(curate.non_validated[\"var_index\"])].copy()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1b0b2b10",
+   "metadata": {},
+   "source": [
+    "Now let's validate the subsetted object:"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "0e139bda",
+   "id": "6ac64695",
    "metadata": {
     "tags": [
      "hide-output"
@@ -342,6 +411,13 @@
    },
    "outputs": [],
    "source": [
+    "curate = ln.Curate.from_anndata(\n",
+    "    adata_validated, \n",
+    "    var_index=bt.Gene.ensembl_gene_id,  # validate var.index against Gene.ensembl_gene_id\n",
+    "    categoricals=categoricals, \n",
+    "    organism=\"human\",\n",
+    ")\n",
+    "\n",
     "curate.validate()"
    ]
   },
@@ -431,7 +507,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.1.-1"
+   "version": "3.10.13"
   },
   "nbproject": {
    "id": "WOK3vP0bNGLx",

{lamindb-0.75.0 → lamindb-0.75.1}/docs/introduction.ipynb

@@ -804,6 +804,7 @@
     "    categoricals={adata.obs.perturbation.name: ln.ULabel.name}, \n",
     "    organism=\"human\",  # specify the organism for the Gene registry\n",
     ")\n",
+    "curate.add_validated_from_var_index()\n",
     "curate.validate()\n",
     "\n",
     "# save curated artifact\n",
@@ -1163,7 +1164,7 @@
     "\n",
     "### Repositories\n",
     "\n",
-    "LaminDB and its plug-ins consist in open-source Python libraries & publicly hosted metadata assets:\n",
+    "LaminDB and its plugins consist in open-source Python libraries & publicly hosted metadata assets:\n",
     "\n",
     "- [lamindb](https://github.com/laminlabs/lamindb): Core package.\n",
     "- [bionty](https://github.com/laminlabs/bionty): Registries for basic biological entities, coupled to public ontologies.\n",

lamindb-0.75.1/docs/storage/vitessce.ipynb

@@ -0,0 +1,203 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Vitessce integration"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "For more comprehensive integration tests, see: https://github.com/laminlabs/lamin-spatial"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!lamin login testuser1\n",
+    "!lamin init --storage \"s3://lamindb-ci/test-vitessce\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import lamindb as ln\n",
+    "import pytest\n",
+    "from vitessce import (\n",
+    "    VitessceConfig,\n",
+    "    Component as cm,\n",
+    "    AnnDataWrapper,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Set up test data:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "pbmc68k = ln.core.datasets.anndata_pbmc68k_reduced()[:100, :200].copy()\n",
+    "zarr_filepath = \"my_test.zarr\"\n",
+    "# write the anndata to a local zarr path\n",
+    "pbmc68k.write_zarr(zarr_filepath)\n",
+    "# create an artifact from the path\n",
+    "dataset_artifact = ln.Artifact(zarr_filepath, description=\"Test dataset\").save()\n",
+    "# this is the where the zarr folder is located on a public S3 bucket\n",
+    "dataset_artifact.path.to_url()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Create a `VitessceConfig` object: "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": [
+     "hide-output"
+    ]
+   },
+   "outputs": [],
+   "source": [
+    "vc = VitessceConfig(schema_version=\"1.0.15\")\n",
+    "vc.add_dataset(name=\"test1\").add_object(\n",
+    "    AnnDataWrapper(\n",
+    "        adata_url=dataset_artifact.path.to_url(),\n",
+    "        obs_embedding_paths=[\"obsm/X_umap\"],\n",
+    "    ),\n",
+    ")\n",
+    "vc.to_dict()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "vitessce_config_artifact = ln.integrations.save_vitessce_config(vc, description=\"View testdata in Vitessce\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# different equivalent ways of testing that the action is attached\n",
+    "assert dataset_artifact._actions.get() == vitessce_config_artifact\n",
+    "assert vitessce_config_artifact._action_targets.get() == dataset_artifact\n",
+    "assert vitessce_config_artifact._actions.first() is None\n",
+    "assert ln.Artifact.filter(_actions=vitessce_config_artifact).one() == dataset_artifact"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "dataset_artifact.delete(permanent=True)\n",
+    "vitessce_config_artifact.delete(permanent=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Test validation within `save_vitessce_config`:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# use a corrupted suffix\n",
+    "vc = VitessceConfig(schema_version=\"1.0.15\")\n",
+    "vc.add_dataset(name=\"test1\").add_object(\n",
+    "    AnnDataWrapper(\n",
+    "        adata_url=dataset_artifact.path.to_url() + \".random\",\n",
+    "        obs_embedding_paths=[\"obsm/X_umap\"],\n",
+    "    ),\n",
+    ")\n",
+    "with pytest.raises(ValueError) as error:\n",
+    "    ln.integrations.save_vitessce_config(vc)\n",
+    "print(error.exconly())\n",
+    "assert error.exconly().startswith(\"ValueError: Suffix should be '.zarr' or one of\")\n",
+    "assert error.exconly().endswith(\".random\")\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# use a corrupted stem_uid\n",
+    "vc = VitessceConfig(schema_version=\"1.0.15\")\n",
+    "vc.add_dataset(name=\"test1\").add_object(\n",
+    "    AnnDataWrapper(\n",
+    "        adata_url=dataset_artifact.path.to_url().replace(dataset_artifact.stem_uid, \"not-a-valid-uid\"),\n",
+    "        obs_embedding_paths=[\"obsm/X_umap\"],\n",
+    "    ),\n",
+    ")\n",
+    "with pytest.raises(ValueError) as error:\n",
+    "    ln.integrations.save_vitessce_config(vc)\n",
+    "print(error.exconly())\n",
+    "assert error.exconly().startswith(\"ValueError: Could not find dataset with stem uid 'not-a-valid-uid' in lamindb\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "ln.setup.delete(\"test-vitessce\", force=True)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "py310",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

{lamindb-0.75.0 → lamindb-0.75.1}/lamindb/__init__.py

@@ -42,7 +42,7 @@ Modules & settings:
 """
 
 # denote a release candidate for 0.1.0 with 0.1rc1, 0.1a1, 0.1b1, etc.
-__version__ = "0.75.0"
+__version__ = "0.75.1"
 
 import os as _os