lamindb 0.48a2__tar.gz → 0.48.1__tar.gz

{lamindb-0.48a2 → lamindb-0.48.1}/.github/workflows/build.yml

@@ -39,17 +39,17 @@ jobs:
           path: ~/.cache/pre-commit
           key: pre-commit-${{ runner.os }}-${{ hashFiles('.pre-commit-config.yaml') }}
       - name: cache postgres
-        if: ${{ matrix.group == 'faq' }}
+        if: ${{ matrix.group == 'faq' || matrix.group == 'unit' }}
         id: cache-postgres
         uses: actions/cache@v3
         with:
           path: ~/postgres.tar
           key: cache-postgres-0
       - name: cache postgres miss
-        if: ${{ steps.cache-postgres.outputs.cache-hit != 'true' && matrix.group == 'faq' }}
+        if: ${{ steps.cache-postgres.outputs.cache-hit != 'true' && matrix.group == 'faq' || matrix.group == 'unit' }}
         run: docker pull postgres:latest && docker image save postgres:latest --output ~/postgres.tar
       - name: cache postgres use
-        if: ${{ steps.cache-postgres.outputs.cache-hit == 'true' && matrix.group == 'faq' }}
+        if: ${{ steps.cache-postgres.outputs.cache-hit == 'true' && matrix.group == 'faq' || matrix.group == 'unit' }}
         run: docker image load --input ~/postgres.tar
       - name: install Python deps
         run: |

{lamindb-0.48a2 → lamindb-0.48.1}/.gitignore

@@ -1,5 +1,4 @@
 # LaminDB
-test-inherit*
 docs/guide/data-validation.ipynb
 docs/guide/knowledge.ipynb
 docs/guide/lnschema-bionty.ipynb
@@ -37,12 +36,12 @@ docs/faq/mydata_postgres/
 docs/guide/myobjects/
 docs/faq/test-run-inputs/
 docs/intro/paradisi05_laminopathic_nuclei.jpg
-test-search0
-test-search1
-test-search2
-test-search3
-test-search4
-test-search5
+docs/guide/figures/
+suo22/
+docs/biology/test-flow/
+docs/biology/test-scrna/
+docs/biology/test-registries/
+docs/biology/test-multimodal/
 # General
 .DS_Store
 
@@ -184,6 +183,7 @@ _build
 *.zip
 *.feather
 *.h5ad
+*.h5mu
 *.parquet
 *.bam
 *.fastq.gz

{lamindb-0.48a2 → lamindb-0.48.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: lamindb
-Version: 0.48a2
+Version: 0.48.1
 Summary: LaminDB: Manage R&D data & analyses.
 Author-email: Lamin Labs <laminlabs@gmail.com>
 Requires-Python: >=3.8
@@ -8,9 +8,9 @@ Description-Content-Type: text/markdown
 Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
-Requires-Dist: lnschema_core==0.39.0
-Requires-Dist: lamindb_setup==0.49.0
-Requires-Dist: lamin_logger==0.8.2
+Requires-Dist: lnschema_core==0.42.2
+Requires-Dist: lamindb_setup==0.49.1
+Requires-Dist: lamin_utils==0.9.3
 Requires-Dist: erdiagram>=0.1.2
 Requires-Dist: rapidfuzz
 Requires-Dist: pydantic[dotenv]
@@ -25,7 +25,7 @@ Requires-Dist: botocore==1.29.76 ; extra == "aws"
 Requires-Dist: urllib3<2 ; extra == "aws"
 Requires-Dist: boto3==1.26.76 ; extra == "aws"
 Requires-Dist: fsspec[s3]==2023.5.0 ; extra == "aws"
-Requires-Dist: lnschema_bionty==0.27.3 ; extra == "bionty"
+Requires-Dist: lnschema_bionty==0.28.2 ; extra == "bionty"
 Requires-Dist: pre-commit ; extra == "dev"
 Requires-Dist: nox ; extra == "dev"
 Requires-Dist: laminci>=0.3 ; extra == "dev"
@@ -73,7 +73,7 @@ Update 2023-06-14:
 
 ## Introduction
 
-LaminDB is a free & open-source Python library to:
+LaminDB is an open-source Python library to:
 
 - Manage files & datasets while tracking provenance across pipelines, notebooks & app uploads.
 - Manage biological registries, ontologies, features & schemas.
@@ -83,7 +83,7 @@ LaminDB is a free & open-source Python library to:
 
 LaminApp is a data management app built on LaminDB. If LaminDB ~ git, LaminApp ~ GitHub.
 
-LaminApp, support, code templates & auto-dispatched integration tests for a BioTech data & analytics platform are currently only available on an enterprise plan. LaminApp is available for your cloud or AWS infrastructure or hosted by us.
+LaminApp, support, code templates & auto-dispatched integration tests for a BioTech data & analytics platform are currently only available on an enterprise plan. LaminApp is available for your cloud infrastructure or hosted by us.
 
 ## Quickstart
 
@@ -257,7 +257,7 @@ pip install lamindb  # basic data management
 You can configure the installation using `extras`, e.g.,
 
 ```shell
-pip install lamindb[jupyter,bionty,fcs,aws]
+pip install 'lamindb[jupyter,bionty,fcs,aws]'
 ```
 
 Supported `extras` are:
@@ -313,6 +313,8 @@ LaminDB consists of the `lamindb` Python package (repository [here](https://gith
 - [lnschema-bionty](https://github.com/laminlabs/lnschema-bionty): Bionty schema, ORMs that are coupled to Bionty's entities.
 - [lnschema-lamin1](https://github.com/laminlabs/lnschema-lamin1): Exemplary configured schema to track samples, treatments, etc.
 - [nbproject](https://github.com/laminlabs/nbproject): Parse metadata from Jupyter notebooks.
+- [lamin-utils](https://github.com/laminlabs/lamin-utils): Utilities for LaminDB and Bionty.
+- [readfcs](https://github.com/laminlabs/readfcs): FCS file reader.
 
 LaminHub & LaminApp are not open-sourced, and neither are templates that model lab operations.
 

{lamindb-0.48a2 → lamindb-0.48.1}/README.md

@@ -19,7 +19,7 @@ Update 2023-06-14:
 
 ## Introduction
 
-LaminDB is a free & open-source Python library to:
+LaminDB is an open-source Python library to:
 
 - Manage files & datasets while tracking provenance across pipelines, notebooks & app uploads.
 - Manage biological registries, ontologies, features & schemas.
@@ -29,7 +29,7 @@ LaminDB is a free & open-source Python library to:
 
 LaminApp is a data management app built on LaminDB. If LaminDB ~ git, LaminApp ~ GitHub.
 
-LaminApp, support, code templates & auto-dispatched integration tests for a BioTech data & analytics platform are currently only available on an enterprise plan. LaminApp is available for your cloud or AWS infrastructure or hosted by us.
+LaminApp, support, code templates & auto-dispatched integration tests for a BioTech data & analytics platform are currently only available on an enterprise plan. LaminApp is available for your cloud infrastructure or hosted by us.
 
 ## Quickstart
 
@@ -203,7 +203,7 @@ pip install lamindb  # basic data management
 You can configure the installation using `extras`, e.g.,
 
 ```shell
-pip install lamindb[jupyter,bionty,fcs,aws]
+pip install 'lamindb[jupyter,bionty,fcs,aws]'
 ```
 
 Supported `extras` are:
@@ -259,6 +259,8 @@ LaminDB consists of the `lamindb` Python package (repository [here](https://gith
 - [lnschema-bionty](https://github.com/laminlabs/lnschema-bionty): Bionty schema, ORMs that are coupled to Bionty's entities.
 - [lnschema-lamin1](https://github.com/laminlabs/lnschema-lamin1): Exemplary configured schema to track samples, treatments, etc.
 - [nbproject](https://github.com/laminlabs/nbproject): Parse metadata from Jupyter notebooks.
+- [lamin-utils](https://github.com/laminlabs/lamin-utils): Utilities for LaminDB and Bionty.
+- [readfcs](https://github.com/laminlabs/readfcs): FCS file reader.
 
 LaminHub & LaminApp are not open-sourced, and neither are templates that model lab operations.
 

lamindb-0.48a2/docs/biology/01-registries.ipynb → lamindb-0.48.1/docs/biology/00-registries.ipynb

@@ -13,9 +13,15 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "If you only work with pre-defined ontologies (public or in-house), [Bionty](https://lamin.ai/docs/bionty/) is sufficient!\n",
+    "Here, you'll learn how to collaborate on biological registries across dry & wetlab.\n",
     "\n",
-    "If you'd like to maintain in-house registries for basic entities along with ontologies, manage them using {mod}`lnschema_bionty`.\n",
+    "What's special is that LaminDB allows you to leverage ontologies for it, based on plug-in {mod}`lnschema_bionty`.\n",
+    "\n",
+    "```{note}\n",
+    "\n",
+    "Registries can anchor dry & wetlab work to help find, access & model data. No more ambiguity about model dimensions and labels!\n",
+    "\n",
+    "```\n",
     "\n",
     "```{toctree}\n",
     ":hidden:\n",
@@ -30,14 +36,16 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Let us start with an instance that has {mod}`lnschema_bionty` mounted:"
+    "Let us start with an instance that has `lnschema_bionty` mounted:"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {
-    "tags": []
+    "tags": [
+     "hide-output"
+    ]
    },
    "outputs": [],
    "source": [
@@ -69,26 +77,20 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Let us first grab a public ontology for cell types:"
+    "Let's consider a public ontology for cell types:"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "metadata": {},
+   "metadata": {
+    "tags": [
+     "hide-output"
+    ]
+   },
    "outputs": [],
    "source": [
-    "celltype_bt = lb.CellType.bionty()  # same as bionty.CellType()\n",
-    "\n",
-    "celltype_bt"
-   ]
-  },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Search a cell type"
+    "celltype_bt = lb.CellType.bionty()  # if you know bionty, is same as bionty.CellType()"
    ]
   },
   {
@@ -97,15 +99,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "celltype_bt.search(\"gamma delta T cell\").head(3)"
-   ]
-  },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "### Look up a cell type with auto-completion"
+    "celltype_bt"
    ]
   },
   {
@@ -113,7 +107,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Create a lookup object:"
+    "We can use it to search cell types:"
    ]
   },
   {
@@ -122,7 +116,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "celltype_bt_lookup = celltype_bt.lookup()"
+    "celltype_bt.search(\"gamma delta T cell\").head(3)"
    ]
   },
   {
@@ -130,16 +124,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "There are 2680 terms in it:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "len(celltype_bt_lookup)"
+    "And we can also use it to look up cell types with auto-complete:"
    ]
   },
   {
@@ -148,8 +133,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gd_tcell = celltype_bt_lookup.gamma_delta_t_cell\n",
-    "gd_tcell"
+    "celltype_bt_lookup = celltype_bt.lookup()\n",
+    "celltype_bt_lookup.gamma_delta_t_cell"
    ]
   },
   {
@@ -165,7 +150,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "You can create a SQL record directly by passing the result of a Bionty search or lookup."
+    "You can create a registry record directly by passing the result of a Bionty search or lookup:"
    ]
   },
   {
@@ -174,7 +159,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "celltype_record = lb.CellType(gd_tcell)\n",
+    "celltype_record = lb.CellType(celltype_bt_lookup.gamma_delta_t_cell)\n",
     "\n",
     "celltype_record"
    ]
@@ -184,59 +169,37 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Note that this ontology has `parents`, and therefore the parent records will also be added to the DB upon save:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "gd_tcell.parents"
-   ]
-  },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Save the record to the DB to seed an in-house ontology:"
+    "When we save this record to the registry, logging informs us that we're also saving parent ontology terms.\n",
+    "\n",
+    "\n",
+    "```{dropdown} Will I always see a 100 parents being saved?\n",
+    "\n",
+    "No, this only happens a single time.\n",
+    "\n",
+    "- If we accidentally save the same record again, lamindb will recognize that the record and all parents are already in the registry.\n",
+    "- If we save another record that has overlapping parents, only new parents will be saved.\n",
+    "\n",
+    "```"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "metadata": {},
+   "metadata": {
+    "tags": [
+     "hide-output"
+    ]
+   },
    "outputs": [],
    "source": [
     "celltype_record.save()"
    ]
   },
   {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "This cell type can now be queried from the DB:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "gd_tcell_record = lb.CellType.select(name=celltype_record.name).one()\n",
-    "gd_tcell_record"
-   ]
-  },
-  {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Access its direct parents:"
+    "View the parents:"
    ]
   },
   {
@@ -245,15 +208,14 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gd_tcell_record.parents.all()"
+    "celltype_record.view_parents()"
    ]
   },
   {
-   "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "View all parents as a graph: (you may specify `distance=` to view a subset of parents)"
+    "Or access them directly:"
    ]
   },
   {
@@ -262,7 +224,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gd_tcell_record.view_parents()"
+    "celltype_record.parents.df()"
    ]
   },
   {
@@ -270,33 +232,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Query for all the parents with a specified distance:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "tags": [
-     "hide-cell"
-    ]
-   },
-   "outputs": [],
-   "source": [
-    "distance = 5\n",
+    "This cell type and all its parents can now be queried & searched in the registry using `lb.CellType.select` and `lb.CellType.search`.\n",
     "\n",
-    "results = []\n",
-    "for d in range(1, distance + 1):\n",
-    "    msg = f\"Depth = {d}\"\n",
-    "    if d == 1:\n",
-    "        condition = \"children__name\"\n",
-    "    else:\n",
-    "        condition = \"children__\" + condition\n",
-    "    records = lb.CellType.select(**{condition: gd_tcell_record.name}).list()\n",
-    "    msg += f\", {len(records)} records found:\"\n",
-    "    print(msg)\n",
-    "    print(records)\n",
-    "    results.append(records)"
+    "Further down the guide, we'll see how this will help us to annotate and validate files & datasets!"
    ]
   },
   {
@@ -468,7 +406,11 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "metadata": {},
+   "metadata": {
+    "tags": [
+     "hide-output"
+    ]
+   },
    "outputs": [],
    "source": [
     "ln.save(cell_types)"