lamindb 1.11.3__tar.gz → 1.12.0__tar.gz

{lamindb-1.11.3 → lamindb-1.12.0}/.gitignore

@@ -200,6 +200,9 @@ venv.bak/
 .dmypy.json
 dmypy.json
 
+# ruff
+.ruff_cache
+
 # Pyre type checker
 .pyre/
 

{lamindb-1.11.3 → lamindb-1.12.0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.3
 Name: lamindb
-Version: 1.11.3
-Summary: A data framework for biology.
+Version: 1.12.0
+Summary: A data lakehouse for biology.
 Author-email: Lamin Labs <open-source@lamin.ai>
 Requires-Python: >=3.10,<3.14
 Description-Content-Type: text/markdown
@@ -10,22 +10,20 @@ Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Requires-Dist: lamin_utils==0.15.0
-Requires-Dist: lamin_cli==1.7.2
-Requires-Dist: lamindb_setup[aws]==1.10.2
-Requires-Dist: bionty>=1.7a1
-Requires-Dist: wetlab>=1.5a1
+Requires-Dist: lamin_cli==1.8.0
+Requires-Dist: lamindb_setup[aws]==1.11.0
+Requires-Dist: bionty==1.8.1
+Requires-Dist: wetlab==1.6.1
 Requires-Dist: nbproject==0.11.1
 Requires-Dist: jupytext
 Requires-Dist: nbconvert>=7.2.1
-Requires-Dist: mistune!=3.1.0
 Requires-Dist: pyyaml
 Requires-Dist: pyarrow
 Requires-Dist: pandera>=0.24.0
 Requires-Dist: typing_extensions!=4.6.0
 Requires-Dist: python-dateutil
 Requires-Dist: pandas>=2.0.0
-Requires-Dist: scipy<1.15.0
-Requires-Dist: anndata>=0.8.0,<=0.12.1
+Requires-Dist: anndata>=0.8.0,<=0.12.2
 Requires-Dist: fsspec
 Requires-Dist: graphviz
 Requires-Dist: psycopg2-binary
@@ -60,8 +58,7 @@ Provides-Extra: zarr
 
 # LaminDB - A data lakehouse for biology
 
-LaminDB is an open-source data lakehouse to enable learning at scale in biology.
-It organizes datasets through validation & annotation and provides data lineage, queryability, and reproducibility on top of [FAIR](https://en.wikipedia.org/wiki/FAIR_data) data.
+LaminDB organizes datasets through validation & annotation and provides data lineage, queryability & reproducibility on top of [FAIR](https://en.wikipedia.org/wiki/FAIR_data) data.
 
 <details>
 <summary>Why?</summary>
@@ -80,17 +77,17 @@ Moreover, it provides context through data lineage -- tracing data and code, sci
 **Highlights.**
 
 - **data lineage:** track inputs & outputs of notebooks, scripts, functions & pipelines with a single line of code
-- **unified infrastructure:** access diverse storage locations (local, S3, GCP, ...), SQL databases (Postgres, SQLite) & ontologies
-- **lakehouse capabilities**: manage, monitor & validate features, labels & dataset schemas; perform distributed queries and batch loading
-- **biological data formats:** validate & annotate formats like `DataFrame`, `AnnData`, `MuData`, ... backed by `parquet`, `zarr`, HDF5, LanceDB, DuckDB, ...
-- **biological entities**: organize experimental metadata & extensible ontologies in registries based on the Django ORM
+- **unified access:** storage locations (local, S3, GCP, ...), SQL databases (Postgres, SQLite) & ontologies
+- **lakehouse**: manage, monitor & validate features, labels & dataset schemas; distributed queries and batch loading
+- **biological formats:** validate & annotate `DataFrame`, `AnnData`, `SpatialData`, ... backed by `parquet`, `zarr`, HDF5, LanceDB, ...
+- **biological entities**: manage experimental metadata & ontologies based on the Django ORM
 - **reproducible & auditable:** auto-version & timestamp execution reports, source code & compute environments, attribute records to users
 - **zero lock-in & scalable:** runs in your infrastructure; is _not_ a client for a rate-limited REST API
 - **extendable:** create custom plug-ins for your own applications based on the Django ecosystem
 - **integrations:** visualization tools like [vitessce](https://docs.lamin.ai/vitessce), workflow managers like [nextflow](https://docs.lamin.ai/nextflow) & [redun](https://docs.lamin.ai/redun), and [other tools](https://docs.lamin.ai/integrations)
 - **production-ready:** used in BigPharma, BioTech, hospitals & top labs
 
-LaminDB can be connected to LaminHub to serve as a [LIMS](https://en.wikipedia.org/wiki/Laboratory_information_management_system) for wetlab scientists, closing the drylab-wetlab feedback loop: [lamin.ai](https://lamin.ai)
+LaminDB can be connected to LaminHub to serve as a [LIMS](https://en.wikipedia.org/wiki/Laboratory_information_management_system) for wetlab scientists, closing the drylab-wetlab feedback loop: [lamin.ai](https://lamin.ai).
 
 ## Docs
 
@@ -155,26 +152,26 @@ artifact.describe()
 
 <img src="https://lamin-site-assets.s3.amazonaws.com/.lamindb/BOTCBgHDAvwglN3U0002.png" width="550">
 
-You can organize datasets with validation & annotation of any kind of metadata to then access them via queries & search. Here is a more [comprehensive example](https://lamin.ai/laminlabs/lamindata/artifact/9K1dteZ6Qx0EXK8g).
+You can organize datasets with validation & annotation of any kind of metadata to then access them via queries & search. Here is a more [comprehensive example](https://lamin.ai/laminlabs/lamindata/artifact/9K1dteZ6Qx0EXK8g):
 
 <img src="https://lamin-site-assets.s3.amazonaws.com/.lamindb/6sofuDVvTANB0f480002.png" width="850">
 
 To annotate an artifact with a label, use:
 
 ```python
-my_experiment = ln.ULabel(name="My experiment").save()  # create a label in the universal label ontology
-artifact.ulabels.add(my_experiment)  # annotate the artifact with the label
+my_experiment = ln.Record(name="My experiment").save()  # create a label in the universal label ontology
+artifact.records.add(my_experiment)  # annotate the artifact with the label
 ```
 
 To query for a set of artifacts, use the `filter()` statement.
 
 ```python
-ln.Artifact.filter(ulabels=my_experiment, suffix=".fasta").to_dataframe()  # query by suffix and the ulabel we just created
+ln.Artifact.filter(records=my_experiment, suffix=".fasta").to_dataframe()  # query by suffix and the ulabel we just created
 ln.Artifact.filter(transform__key="create-fasta.py").to_dataframe()  # query by the name of the script we just ran
 ```
 
 If you have a structured dataset like a `DataFrame`, an `AnnData`, or another array, you can validate the content of the dataset (and parse annotations).
-Here is an example for a dataframe: [docs.lamin.ai/introduction#validate-an-artifact](https://docs.lamin.ai/introduction#validate-an-artifact).
+Here is [an example for a dataframe](https://docs.lamin.ai/tutorial#validate-an-artifact).
 
 With a large body of validated datasets, you can then access data through distributed queries & batch streaming, see here: [docs.lamin.ai/arrays](https://docs.lamin.ai/arrays).
 

{lamindb-1.11.3 → lamindb-1.12.0}/README.md

@@ -7,8 +7,7 @@
 
 # LaminDB - A data lakehouse for biology
 
-LaminDB is an open-source data lakehouse to enable learning at scale in biology.
-It organizes datasets through validation & annotation and provides data lineage, queryability, and reproducibility on top of [FAIR](https://en.wikipedia.org/wiki/FAIR_data) data.
+LaminDB organizes datasets through validation & annotation and provides data lineage, queryability & reproducibility on top of [FAIR](https://en.wikipedia.org/wiki/FAIR_data) data.
 
 <details>
 <summary>Why?</summary>
@@ -27,17 +26,17 @@ Moreover, it provides context through data lineage -- tracing data and code, sci
 **Highlights.**
 
 - **data lineage:** track inputs & outputs of notebooks, scripts, functions & pipelines with a single line of code
-- **unified infrastructure:** access diverse storage locations (local, S3, GCP, ...), SQL databases (Postgres, SQLite) & ontologies
-- **lakehouse capabilities**: manage, monitor & validate features, labels & dataset schemas; perform distributed queries and batch loading
-- **biological data formats:** validate & annotate formats like `DataFrame`, `AnnData`, `MuData`, ... backed by `parquet`, `zarr`, HDF5, LanceDB, DuckDB, ...
-- **biological entities**: organize experimental metadata & extensible ontologies in registries based on the Django ORM
+- **unified access:** storage locations (local, S3, GCP, ...), SQL databases (Postgres, SQLite) & ontologies
+- **lakehouse**: manage, monitor & validate features, labels & dataset schemas; distributed queries and batch loading
+- **biological formats:** validate & annotate `DataFrame`, `AnnData`, `SpatialData`, ... backed by `parquet`, `zarr`, HDF5, LanceDB, ...
+- **biological entities**: manage experimental metadata & ontologies based on the Django ORM
 - **reproducible & auditable:** auto-version & timestamp execution reports, source code & compute environments, attribute records to users
 - **zero lock-in & scalable:** runs in your infrastructure; is _not_ a client for a rate-limited REST API
 - **extendable:** create custom plug-ins for your own applications based on the Django ecosystem
 - **integrations:** visualization tools like [vitessce](https://docs.lamin.ai/vitessce), workflow managers like [nextflow](https://docs.lamin.ai/nextflow) & [redun](https://docs.lamin.ai/redun), and [other tools](https://docs.lamin.ai/integrations)
 - **production-ready:** used in BigPharma, BioTech, hospitals & top labs
 
-LaminDB can be connected to LaminHub to serve as a [LIMS](https://en.wikipedia.org/wiki/Laboratory_information_management_system) for wetlab scientists, closing the drylab-wetlab feedback loop: [lamin.ai](https://lamin.ai)
+LaminDB can be connected to LaminHub to serve as a [LIMS](https://en.wikipedia.org/wiki/Laboratory_information_management_system) for wetlab scientists, closing the drylab-wetlab feedback loop: [lamin.ai](https://lamin.ai).
 
 ## Docs
 
@@ -102,25 +101,25 @@ artifact.describe()
 
 <img src="https://lamin-site-assets.s3.amazonaws.com/.lamindb/BOTCBgHDAvwglN3U0002.png" width="550">
 
-You can organize datasets with validation & annotation of any kind of metadata to then access them via queries & search. Here is a more [comprehensive example](https://lamin.ai/laminlabs/lamindata/artifact/9K1dteZ6Qx0EXK8g).
+You can organize datasets with validation & annotation of any kind of metadata to then access them via queries & search. Here is a more [comprehensive example](https://lamin.ai/laminlabs/lamindata/artifact/9K1dteZ6Qx0EXK8g):
 
 <img src="https://lamin-site-assets.s3.amazonaws.com/.lamindb/6sofuDVvTANB0f480002.png" width="850">
 
 To annotate an artifact with a label, use:
 
 ```python
-my_experiment = ln.ULabel(name="My experiment").save()  # create a label in the universal label ontology
-artifact.ulabels.add(my_experiment)  # annotate the artifact with the label
+my_experiment = ln.Record(name="My experiment").save()  # create a label in the universal label ontology
+artifact.records.add(my_experiment)  # annotate the artifact with the label
 ```
 
 To query for a set of artifacts, use the `filter()` statement.
 
 ```python
-ln.Artifact.filter(ulabels=my_experiment, suffix=".fasta").to_dataframe()  # query by suffix and the ulabel we just created
+ln.Artifact.filter(records=my_experiment, suffix=".fasta").to_dataframe()  # query by suffix and the ulabel we just created
 ln.Artifact.filter(transform__key="create-fasta.py").to_dataframe()  # query by the name of the script we just ran
 ```
 
 If you have a structured dataset like a `DataFrame`, an `AnnData`, or another array, you can validate the content of the dataset (and parse annotations).
-Here is an example for a dataframe: [docs.lamin.ai/introduction#validate-an-artifact](https://docs.lamin.ai/introduction#validate-an-artifact).
+Here is [an example for a dataframe](https://docs.lamin.ai/tutorial#validate-an-artifact).
 
 With a large body of validated datasets, you can then access data through distributed queries & batch streaming, see here: [docs.lamin.ai/arrays](https://docs.lamin.ai/arrays).

{lamindb-1.11.3 → lamindb-1.12.0}/docs/arrays.ipynb

@@ -18,31 +18,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "We saw how LaminDB allows to query & search across artifacts & collections using registries: {doc}`registries`.\n",
+    "We saw how LaminDB allows to query & search across artifacts using registries: {doc}`registries`.\n",
     "\n",
-    "Let us now look at the following case:\n",
-    "\n",
-    "```\n",
-    "# get a lookup for labels\n",
-    "ulabels = ln.ULabel.lookup()\n",
-    "# query a parquet file matching an \"setosa\"\n",
-    "df = ln.Artifact.filter(ulabels=ulabels.setosa, suffix=\".suffix\").first().load()\n",
-    "# query all observations in the DataFrame matching \"setosa\"\n",
-    "df_setosa = df.loc[:, df.iris_organism_name == ulabels.setosa.name]  \n",
-    "```\n",
-    "\n",
-    "Because the artifact was validated, querying the `DataFrame` is guaranteed to succeed!\n",
-    "\n",
-    "Such within-collection queries are also possible for cloud-backed collections using [DuckDB](https://duckdb.org),\n",
-    "[TileDB](https://tiledb.com), [zarr](https://zarr.readthedocs.io/en/stable), [HDF5](https://docs.h5py.org/en/stable),\n",
-    "[parquet](https://parquet.apache.org), and other storage backends.\n",
-    "\n",
-    "- For a use case with TileDB, see: {doc}`docs:cellxgene`\n",
-    "- For a use case with DuckDB, see: {doc}`docs:rxrx`\n",
-    "\n",
-    "In this notebook, we show how to subset an `AnnData` and generic `HDF5` and `zarr` collections accessed in the cloud.\n",
-    "\n",
-    "Let us create a remote instance for testing."
+    "Let us now query the datasets in storage themselves. Here, we show how to subset an `AnnData` and generic `HDF5` and `zarr` collections accessed in the cloud."
    ]
   },
   {
@@ -55,6 +33,7 @@
    },
    "outputs": [],
    "source": [
+    "# replace with your username and S3 bucket\n",
     "!lamin login testuser1\n",
     "!lamin init --storage s3://lamindb-ci/test-arrays"
    ]
@@ -80,7 +59,7 @@
     "import numpy as np\n",
     "import zarr\n",
     "\n",
-    "ln.track(\"hsRyWJggf2Ca\")"
+    "ln.track()"
    ]
   },
   {