lamindb 0.74.0__tar.gz → 0.74.2__tar.gz

{lamindb-0.74.0 → lamindb-0.74.2}/.github/workflows/build.yml

@@ -4,7 +4,6 @@ on:
   push:
     branches: [release]
   pull_request:
-    branches: [release, main, staging]
 
 jobs:
   test:
@@ -55,7 +54,7 @@ jobs:
       - name: cache postgres use
         if: ${{ steps.cache-postgres.outputs.cache-hit == 'true' }}
         run: docker image load --input ~/postgres.tar
-      - run: pip install -U laminci
+      - run: pip install "laminci@git+https://x-access-token:${{ secrets.LAMIN_BUILD_DOCS }}@github.com/laminlabs/laminci"
       - name: install postgres
         if: ${{ matrix.group == 'faq' }}
         run: sudo apt-get install libpq-dev

lamindb-0.74.2/.github/workflows/doc-changes.yml

@@ -0,0 +1,23 @@
+name: doc-changes
+
+on:
+  pull_request_target:
+    branches:
+      - main
+    types:
+      - closed
+
+jobs:
+  doc-changes:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - run: pip install "laminci[doc-changes]@git+https://x-access-token:${{ secrets.LAMIN_BUILD_DOCS }}@github.com/laminlabs/laminci"
+      - run: laminci doc-changes
+        env:
+          repo_token: ${{ secrets.GITHUB_TOKEN }}
+          docs_token: ${{ secrets.LAMIN_BUILD_DOCS }}
+          changelog_file: lamin-docs/docs/changelog/soon/lamindb.md

{lamindb-0.74.0 → lamindb-0.74.2}/.gitignore

@@ -19,6 +19,7 @@ bfx-output/
 scrnaseq-cellranger/
 scrnaseq-cellranger-schmidt22/
 cellranger_run_001
+docs/test-annotate-flexible/
 __MACOSX/
 docs/lamindb.*
 lamin_sphinx
@@ -60,6 +61,7 @@ test-search3
 test-search4
 test-search5
 default_storage
+test.ipynb
 
 # General
 .DS_Store
@@ -206,6 +208,7 @@ _build
 *.parquet
 *.bam
 *.fastq.gz
+*.pt
 
 # Pycharm
 .idea

{lamindb-0.74.0 → lamindb-0.74.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: lamindb
-Version: 0.74.0
+Version: 0.74.2
 Summary: A data framework for biology.
 Author-email: Lamin Labs <open-source@lamin.ai>
 Requires-Python: >=3.8
@@ -9,24 +9,22 @@ Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
-Requires-Dist: lnschema_core==0.70.0
-Requires-Dist: lamindb_setup==0.73.2
+Requires-Dist: lnschema_core==0.70.5
+Requires-Dist: lamindb_setup==0.74.1
 Requires-Dist: lamin_utils==0.13.2
-Requires-Dist: lamin_cli==0.14.0
+Requires-Dist: lamin_cli==0.15.0
 Requires-Dist: rapidfuzz
 Requires-Dist: pyarrow
 Requires-Dist: typing_extensions!=4.6.0
 Requires-Dist: python-dateutil
-Requires-Dist: anndata>=0.8.0,<0.10.8
+Requires-Dist: anndata>=0.8.0,<=0.10.8
 Requires-Dist: scipy<1.13.0rc1
 Requires-Dist: fsspec
 Requires-Dist: pandas
 Requires-Dist: graphviz
 Requires-Dist: psycopg2-binary
-Requires-Dist: psutil
 Requires-Dist: lamindb_setup[aws] ; extra == "aws"
-Requires-Dist: bionty==0.44.0 ; extra == "bionty"
-Requires-Dist: pandas<2 ; extra == "dev"
+Requires-Dist: bionty==0.44.2 ; extra == "bionty"
 Requires-Dist: pre-commit ; extra == "dev"
 Requires-Dist: nox ; extra == "dev"
 Requires-Dist: laminci>=0.3 ; extra == "dev"
@@ -59,7 +57,7 @@ Provides-Extra: zarr
 
 # LaminDB - A data framework for biology
 
-- Manage storage & databases with a unified Python API ("lakehouse").
+- Manage data & metadata with a unified Python API ("lakehouse").
 - Track data lineage across notebooks & pipelines.
 - Integrate registries for experimental metadata & in-house ontologies.
 - Validate, standardize & annotate.

{lamindb-0.74.0 → lamindb-0.74.2}/README.md

@@ -4,7 +4,7 @@
 
 # LaminDB - A data framework for biology
 
-- Manage storage & databases with a unified Python API ("lakehouse").
+- Manage data & metadata with a unified Python API ("lakehouse").
 - Track data lineage across notebooks & pipelines.
 - Integrate registries for experimental metadata & in-house ontologies.
 - Validate, standardize & annotate.

lamindb-0.74.0/docs/reference.md → lamindb-0.74.2/docs/api.md

@@ -1,4 +1,4 @@
-# Reference
+# API
 
 <meta http-equiv="Refresh" content="0; url=./lamindb.html" />
 

lamindb-0.74.0/docs/data.ipynb → lamindb-0.74.2/docs/arrays.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "[![Jupyter Notebook](https://img.shields.io/badge/Source%20on%20GitHub-orange)](https://github.com/laminlabs/lamindb/blob/main/docs/data.ipynb)"
+    "[![Jupyter Notebook](https://img.shields.io/badge/Source%20on%20GitHub-orange)](https://github.com/laminlabs/lamindb/blob/main/docs/arrays.ipynb)"
    ]
   },
   {
@@ -20,7 +20,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We saw how LaminDB allows to query & search across artifacts & collections using registries: {doc}`meta`.\n",
+    "We saw how LaminDB allows to query & search across artifacts & collections using registries: {doc}`records`.\n",
     "\n",
     "Let us now look at the following case:\n",
     "\n",
@@ -134,7 +134,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "adata = artifact.backed()"
+    "adata = artifact.open()"
    ]
   },
   {
@@ -265,7 +265,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "backed = artifact.backed()"
+    "backed = artifact.open()"
    ]
   },
   {
@@ -325,7 +325,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.13"
+   "version": "3.11.7"
   },
   "nbproject": {
    "id": "YVUCtH4GfQOy",

{lamindb-0.74.0 → lamindb-0.74.2}/docs/bio-registries.ipynb

@@ -20,30 +20,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Registries manage the formalized knowledge & experimental design that anchor dry & wetlab work.\n",
-    "\n",
-    "In LaminDB, registries are standard SQL tables, equipped with [mechanisms that avoid typos & duplicated data](/faq/idempotency).\n",
-    "\n",
-    "In addition, LaminDB makes it easy to import records from public biological ontologies, based on plug-in {mod}`bionty`.\n",
-    "\n",
-    "In this notebook, you'll see how to manage an in-house ontology anchored in public knowledge.\n",
-    "\n",
-    "(If you also manage experimental design through registries, you can access all metadata through one API and store it in one simple SQL database.)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## Setup"
-   ]
-  },
-  {
-   "attachments": {},
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Let us create an instance that has {mod}`bionty` mounted, which extends to include biological registries:"
+    "With plug-in {mod}`bionty`, it becomes easy to import records from public biological ontologies."
    ]
   },
   {
@@ -56,27 +33,15 @@
    },
    "outputs": [],
    "source": [
+    "#! pip install 'lamindb[bionty]'\n",
     "!lamin init --storage ./test-registries --schema bionty"
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import lamindb as ln\n",
-    "import bionty as bt\n",
-    "\n",
-    "ln.connect(\"test-registries\")\n",
-    "ln.settings.verbosity = \"info\""
-   ]
-  },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Let's pre-populate the cell type registry with a few records:"
+    "Let's pre-populate the {class}`bionty.Organism` and {class}`bionty.CellType` registry with a few records:"
    ]
   },
   {
@@ -89,6 +54,9 @@
    },
    "outputs": [],
    "source": [
+    "import lamindb as ln\n",
+    "import bionty as bt\n",
+    "\n",
     "bt.Organism.from_public(name=\"human\").save()\n",
     "bt.CellType.from_public(name=\"T cell\").save()\n",
     "bt.CellType(name=\"my T cell subtype\").save()"
@@ -107,26 +75,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We start with a public ontology for cell types.\n",
-    "\n",
-    "`.public()` gives you access to the `ReferenceTable`, a class for accessing public ontologies.\n",
-    "\n",
-    "`ReferenceTable` provides simple access to standardized public ontologies that Lamin hosts for reliable and performant access. For most `ReferenceTable` objects, you can access the underlying ontology through [Pronto](https://github.com/althonos/pronto).\n",
-    "\n",
-    "Let's create a `ReferenceTable` object:"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "tags": [
-     "hide-output"
-    ]
-   },
-   "outputs": [],
-   "source": [
-    "public = bt.CellType.public()"
+    "Consider a public ontology for cell types: `.public()` returns a {class}`bionty.core.PublicOntology` object for accessing a public ontology."
    ]
   },
   {
@@ -135,6 +84,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "public = bt.CellType.public()\n",
     "public"
    ]
   },
@@ -164,7 +114,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "And we can also use it to look up cell types with auto-complete:"
+    "Or to look up cell types with auto-complete:"
    ]
   },
   {
@@ -215,7 +165,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "(Alternatively, we could construct the gamma delta T cell via {meth}`~bionty.core.BioRegistry.from_public`, which is synonyms-aware.)"
+    "Alternatively, we can construct the gamma delta T cell via {meth}`~bionty.core.BioRecord.from_public`, which is synonyms-aware:"
    ]
   },
   {
@@ -236,7 +186,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "When we save this record to the registry, logging informs us that we're also saving parent ontological terms:"
+    "When we save this record to the registry, logging informs us that we're also saving parent records:"
    ]
   },
   {
@@ -306,7 +256,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "You can construct custom hierarchies of terms by specifying parents:"
+    "You can construct custom hierarchies of records:"
    ]
   },
   {
@@ -316,15 +266,7 @@
    "outputs": [],
    "source": [
     "my_celltype = bt.CellType.filter(name=\"my T cell subtype\").one()\n",
-    "my_celltype.parents.add(gdt_cell)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
+    "my_celltype.parents.add(gdt_cell)\n",
     "gdt_cell.view_parents(distance=2, with_children=True)"
    ]
   },
@@ -333,7 +275,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "This cell type and all its parents can now be queried & searched in the registry using `bionty.CellType.filter` and `bionty.CellType.search`."
+    "This cell type and all its parents can now be queried & searched in the registry via `bt.CellType.filter()` and `bt.CellType.search()`."
    ]
   },
   {
@@ -374,7 +316,7 @@
    "source": [
     "We'd like to load the corresponding records in our in-house ontology to annotate a dataset.\n",
     "\n",
-    "To this end, you'll typically use {class}`~lamindb.core.Registry.from_values`, which will both validate & load records that match the values."
+    "To this end, you'll typically use {class}`~lamindb.core.Record.from_values`, which will both validate & load records that match the values."
    ]
   },
   {
@@ -404,10 +346,10 @@
     "\n",
     "1. If registry records match the values, load these records\n",
     "2. If values match synonyms of registry records, load these records\n",
-    "3. (`bionty`-only) If no record in the registry matches, attempt to load records from a public reference\n",
-    "4. (`bionty`-only) Same as 3. but based on synonyms\n",
+    "3. If no record in the registry matches, attempt to load records from a public ontology\n",
+    "4. Same as 3. but based on synonyms\n",
     "\n",
-    "No records will be returned if input field values aren't mappable.\n",
+    "No records will be returned if all 4 look ups are unsuccessful.\n",
     "\n",
     "Example:\n",
     "\n",
@@ -431,6 +373,16 @@
     "my_celltype.save()\n",
     "```\n",
     "\n",
+    "Sometimes, it's useful to treat validated records differently from non-validated records. Here is a way:\n",
+    "\n",
+    "```\n",
+    "original_values = [\"gut\", \"gut2\"]\n",
+    "validated_status = bt.Tissue.validate(original_values)\n",
+    "validated_values = [value for value, validated in zip(original_values, validated_status) if validated]\n",
+    "records_from_validated_values = bt.Tissue.from_values(validated_values)\n",
+    "ln.save(records_from_validated_values)\n",
+    "```\n",
+    "\n",
     ":::\n"
    ]
   },
@@ -473,7 +425,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "If we're happy with `cell_types` records, we save them to the registry:"
+    "If we're happy with the cell type records, we save them to the registry:"
    ]
   },
   {
@@ -494,7 +446,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Now, let's inspect our in-house registry:"
+    "Now, let's look at our in-house registry:"
    ]
   },
   {
@@ -557,7 +509,6 @@
    "source": [
     "cell_types = bt.CellType.lookup()\n",
     "hsc_record = cell_types.hematopoietic_stem_cell\n",
-    "\n",
     "hsc_record"
    ]
   },
@@ -718,7 +669,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Manage registries across organism"
+    "## Manage registries across organisms"
    ]
   },
   {
@@ -770,15 +721,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "bt.settings.organism = \"mouse\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
+    "bt.settings.organism = \"mouse\"\n",
     "bt.Gene.from_public(symbol=\"Ap5b1\")"
    ]
   },
@@ -906,8 +849,7 @@
    "outputs": [],
    "source": [
     "# clean up test instance\n",
-    "!lamin delete --force test-registries\n",
-    "!rm -r test-registries"
+    "!lamin delete --force test-registries"
    ]
   }
  ],

lamindb-0.74.2/docs/changelog.md

@@ -0,0 +1,3 @@
+# Changelog
+
+Actual content in lamin-docs.