lamindb 0.49.3__tar.gz → 0.50.1__tar.gz

{lamindb-0.49.3 → lamindb-0.50.1}/.github/workflows/build.yml

@@ -76,7 +76,8 @@ jobs:
         with:
           name: coverage--${{ matrix.group }}
           path: .coverage
-      - run: mv ./docs/lnschema-bionty.ipynb ./docs/biology/
+      # manually move two notebooks
+      - run: mv ./docs/tutorial1.ipynb ./docs/guide/
       - name: upload docs
         if: ${{ matrix.group == 'guide' || matrix.group == 'biology' || matrix.group == 'faq' || matrix.group == 'storage' }}
         uses: actions/upload-artifact@v2

{lamindb-0.49.3 → lamindb-0.50.1}/.gitignore

@@ -3,6 +3,7 @@ docs/guide/data-validation.ipynb
 docs/guide/knowledge.ipynb
 docs/guide/lnschema-bionty.ipynb
 docs/guide/lnschema-core.ipynb
+docs/paradisi05_laminopathic_nuclei.jpg
 lnschema_bionty_docs/
 lnschema_core_docs/
 _build

{lamindb-0.49.3 → lamindb-0.50.1}/.pre-commit-config.yaml

@@ -73,4 +73,4 @@ repos:
       - id: pydocstyle
         exclude: lamindb/schema/dev/__init__.py
         args: # google style + __init__, see http://www.pydocstyle.org/en/stable/error_codes.html
-          - --ignore=D100,D101,D102,D103,D105,D106,D107,D203,D204,D213,D215,D400,D401,D403,D404,D406,D407,D408,D409,D412,D413,D415,D418,D104,D417
+          - --ignore=D100,D101,D102,D103,D105,D106,D107,D203,D204,D213,D214,D215,D400,D401,D403,D404,D406,D407,D408,D409,D412,D413,D415,D418,D104,D417

{lamindb-0.49.3 → lamindb-0.50.1}/PKG-INFO

@@ -1,16 +1,16 @@
 Metadata-Version: 2.1
 Name: lamindb
-Version: 0.49.3
-Summary: Open-source data lake & feature store for biology.
+Version: 0.50.1
+Summary: Open-source data platform for biology.
 Author-email: Lamin Labs <laminlabs@gmail.com>
 Requires-Python: >=3.8
 Description-Content-Type: text/markdown
 Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
-Requires-Dist: lnschema_core==0.43.3
-Requires-Dist: lamindb_setup==0.49.4
-Requires-Dist: lamin_utils==0.9.3
+Requires-Dist: lnschema_core==0.44.3
+Requires-Dist: lamindb_setup==0.49.6
+Requires-Dist: lamin_utils==0.9.6
 Requires-Dist: erdiagram>=0.1.2
 Requires-Dist: rapidfuzz
 Requires-Dist: pydantic[dotenv]
@@ -24,7 +24,7 @@ Requires-Dist: botocore==1.29.76 ; extra == "aws"
 Requires-Dist: urllib3<2 ; extra == "aws"
 Requires-Dist: boto3==1.26.76 ; extra == "aws"
 Requires-Dist: fsspec[s3]==2023.5.0 ; extra == "aws"
-Requires-Dist: lnschema_bionty==0.29.0 ; extra == "bionty"
+Requires-Dist: lnschema_bionty==0.29.2 ; extra == "bionty"
 Requires-Dist: pre-commit ; extra == "dev"
 Requires-Dist: nox ; extra == "dev"
 Requires-Dist: laminci>=0.3 ; extra == "dev"
@@ -39,7 +39,7 @@ Requires-Dist: fsspec[gs]==2023.5.0 ; extra == "gcp"
 Requires-Dist: nbproject==0.9.0 ; extra == "jupyter"
 Requires-Dist: nbproject==0.9.0 ; extra == "nbproject"
 Requires-Dist: psycopg2-binary ; extra == "postgres"
-Requires-Dist: zarr ; extra == "zarr"
+Requires-Dist: zarr>=2.16.0 ; extra == "zarr"
 Project-URL: Home, https://github.com/laminlabs/lamindb
 Provides-Extra: aws
 Provides-Extra: bionty
@@ -58,27 +58,27 @@ Provides-Extra: zarr
 
 # LaminDB
 
-Open-source data lake & feature store for biology.
+Open-source data platform for biology.
 
 ```{warning}
 
-Public beta: Currently only recommended for collaborators as we still make breaking changes.
+Public beta: Close to having converged a stable API, but some breaking changes might still occur.
 
-Update 2023-06-14:
-
-- We completed a major migration from SQLAlchemy/SQLModel to Django, available in 0.42.0.
-- The last version before the migration is 0.41.2.
 ```
 
 ## Introduction
 
-LaminDB is an open-source Python library to:
+LaminDB is an open-source Python library to manage files & datasets while
+
+- tracking provenance across pipelines, notebooks & app uploads
+- validating & linking data batches using biological registries & ontologies
+
+You can
 
-- Manage files & datasets while tracking provenance across pipelines, notebooks & app uploads.
-- Manage biological registries, ontologies, features & schemas.
-- Enhance integrity through built-in data validation and [idempotent](https://lamin.ai/docs/faq/idempotency), [ACID](https://lamin.ai/docs/faq/acid) operations.
+- Manage features & labels schema-less or schema-full.
 - Query, search, look up, save, load and stream with one API.
 - Collaborate across a mesh of LaminDB instances.
+- Enjoy [idempotent](https://lamin.ai/docs/faq/idempotency) & [ACID](https://lamin.ai/docs/faq/acid) operations.
 
 LaminApp is a data management app built on LaminDB. If LaminDB ~ git, LaminApp ~ GitHub.
 
@@ -101,17 +101,22 @@ import lamindb as ln
 # import lnschema_bionty as lb # optional, for bionty schema
 ```
 
-### Manage data objects
+### Manage files and data objects
 
 ```python
-# Store a DataFrame object
+# Store and register a file
+ln.File("s3://my-bucket/images/image001.jpg").save()  # or a local path
+
+# Store and register a DataFrame object
 df = pd.DataFrame({"feat1": [1, 2], "feat2": [3, 4]})  # AnnData works, too
 ln.File(df, description="Data batch 1").save()  # create a File object and save/upload it
 
 # To find it, if you don't have specific metadata in mind, run a search
 ln.File.search("batch 1")
-# Or filter (under-the-hood, you have the full power of SQL to query)
+# Or query (under-the-hood, you have the full power of SQL to query)
 file = ln.File.filter(description="Data batch 1").one()  # get exactly one result
+# Query by `key` (the relative path within your storage)
+ln.File.filter(key__startswith="images/").df()  # get query results as a DataFrame
 
 # Load a file back into memory
 df = file.load()
@@ -119,27 +124,19 @@ df = file.load()
 backed = file.backed()  # currently works for AnnData, zarr, HDF5, not yet for DataFrame
 ```
 
-### Manage files
-
-```python
-# Store a file
-file = ln.File("s3://my-bucket/images/image001.jpg")  # or a local path
-file.save()  # register the file
-
-# Query by `key` (the relative path within your storage) and load into memory
-file.filter(key__startswith="images/").df()  # all files in folder "images/" in default storage
-```
-
 ### Auto-complete categoricals and search
 
 ```python
-# When you're unsure about spellings, use a lookup object:
-users = ln.User.lookup()
+# When you're unsure about spellings, use a lookup object
+users = ln.User.lookup() # by default uses handle field
 ln.File.filter(created_by=users.lizlemon)
+users = ln.User.lookup("name") # a lookup object of the name field
+users.liz_lemon
 
 # Or search
-ln.User.search("liz lemon", field="name")
+ln.User.search("liz lemon") # by default searches the handle field
 user = ln.User.search("liz lemon", return_queryset=True).first() # grab the top search result as a record
+ln.User.search("liz lemon", field=["name", "handle"]) # search against multiple fields
 ```
 
 ### Track & query data lineage
@@ -161,11 +158,12 @@ file.view_lineage()
 Track a Jupyter Notebook:
 
 ```python
+# my-analysis.ipynb
 ln.track()  # auto-detect & save notebook metadata
 ln.File("my_artifact.parquet").save()  # this file is now aware that it was saved in this notebook
 ```
 
-When you query the file, later on, you'll know from which notebook it came:
+When you query the file, later on, you'll know which notebook it came from:
 
 ```python
 file = ln.File.filter(description="my_artifact.parquet").one()  # query for a file
@@ -314,7 +312,7 @@ We do _not_ store any of your data, but only basic metadata about you (email add
 
 LaminDB builds semantics of R&D and biology onto well-established tools:
 
-- SQLite & Postgres for SQL databases using Django ORM (previously: SQLModel)
+- SQLite & Postgres for SQL databases using Django Registry (previously: SQLModel)
 - S3, GCP & local storage for object storage using fsspec
 - Configurable storage formats: pyarrow, anndata, zarr, etc.
 - Biological knowledge sources & ontologies: see [Bionty](https://lamin.ai/docs/bionty)

{lamindb-0.49.3 → lamindb-0.50.1}/README.md

@@ -5,27 +5,27 @@
 
 # LaminDB
 
-Open-source data lake & feature store for biology.
+Open-source data platform for biology.
 
 ```{warning}
 
-Public beta: Currently only recommended for collaborators as we still make breaking changes.
+Public beta: Close to having converged a stable API, but some breaking changes might still occur.
 
-Update 2023-06-14:
-
-- We completed a major migration from SQLAlchemy/SQLModel to Django, available in 0.42.0.
-- The last version before the migration is 0.41.2.
 ```
 
 ## Introduction
 
-LaminDB is an open-source Python library to:
+LaminDB is an open-source Python library to manage files & datasets while
+
+- tracking provenance across pipelines, notebooks & app uploads
+- validating & linking data batches using biological registries & ontologies
+
+You can
 
-- Manage files & datasets while tracking provenance across pipelines, notebooks & app uploads.
-- Manage biological registries, ontologies, features & schemas.
-- Enhance integrity through built-in data validation and [idempotent](https://lamin.ai/docs/faq/idempotency), [ACID](https://lamin.ai/docs/faq/acid) operations.
+- Manage features & labels schema-less or schema-full.
 - Query, search, look up, save, load and stream with one API.
 - Collaborate across a mesh of LaminDB instances.
+- Enjoy [idempotent](https://lamin.ai/docs/faq/idempotency) & [ACID](https://lamin.ai/docs/faq/acid) operations.
 
 LaminApp is a data management app built on LaminDB. If LaminDB ~ git, LaminApp ~ GitHub.
 
@@ -48,17 +48,22 @@ import lamindb as ln
 # import lnschema_bionty as lb # optional, for bionty schema
 ```
 
-### Manage data objects
+### Manage files and data objects
 
 ```python
-# Store a DataFrame object
+# Store and register a file
+ln.File("s3://my-bucket/images/image001.jpg").save()  # or a local path
+
+# Store and register a DataFrame object
 df = pd.DataFrame({"feat1": [1, 2], "feat2": [3, 4]})  # AnnData works, too
 ln.File(df, description="Data batch 1").save()  # create a File object and save/upload it
 
 # To find it, if you don't have specific metadata in mind, run a search
 ln.File.search("batch 1")
-# Or filter (under-the-hood, you have the full power of SQL to query)
+# Or query (under-the-hood, you have the full power of SQL to query)
 file = ln.File.filter(description="Data batch 1").one()  # get exactly one result
+# Query by `key` (the relative path within your storage)
+ln.File.filter(key__startswith="images/").df()  # get query results as a DataFrame
 
 # Load a file back into memory
 df = file.load()
@@ -66,27 +71,19 @@ df = file.load()
 backed = file.backed()  # currently works for AnnData, zarr, HDF5, not yet for DataFrame
 ```
 
-### Manage files
-
-```python
-# Store a file
-file = ln.File("s3://my-bucket/images/image001.jpg")  # or a local path
-file.save()  # register the file
-
-# Query by `key` (the relative path within your storage) and load into memory
-file.filter(key__startswith="images/").df()  # all files in folder "images/" in default storage
-```
-
 ### Auto-complete categoricals and search
 
 ```python
-# When you're unsure about spellings, use a lookup object:
-users = ln.User.lookup()
+# When you're unsure about spellings, use a lookup object
+users = ln.User.lookup() # by default uses handle field
 ln.File.filter(created_by=users.lizlemon)
+users = ln.User.lookup("name") # a lookup object of the name field
+users.liz_lemon
 
 # Or search
-ln.User.search("liz lemon", field="name")
+ln.User.search("liz lemon") # by default searches the handle field
 user = ln.User.search("liz lemon", return_queryset=True).first() # grab the top search result as a record
+ln.User.search("liz lemon", field=["name", "handle"]) # search against multiple fields
 ```
 
 ### Track & query data lineage
@@ -108,11 +105,12 @@ file.view_lineage()
 Track a Jupyter Notebook:
 
 ```python
+# my-analysis.ipynb
 ln.track()  # auto-detect & save notebook metadata
 ln.File("my_artifact.parquet").save()  # this file is now aware that it was saved in this notebook
 ```
 
-When you query the file, later on, you'll know from which notebook it came:
+When you query the file, later on, you'll know which notebook it came from:
 
 ```python
 file = ln.File.filter(description="my_artifact.parquet").one()  # query for a file
@@ -261,7 +259,7 @@ We do _not_ store any of your data, but only basic metadata about you (email add
 
 LaminDB builds semantics of R&D and biology onto well-established tools:
 
-- SQLite & Postgres for SQL databases using Django ORM (previously: SQLModel)
+- SQLite & Postgres for SQL databases using Django Registry (previously: SQLModel)
 - S3, GCP & local storage for object storage using fsspec
 - Configurable storage formats: pyarrow, anndata, zarr, etc.
 - Biological knowledge sources & ontologies: see [Bionty](https://lamin.ai/docs/bionty)

{lamindb-0.49.3 → lamindb-0.50.1}/docs/biology/00-registries.ipynb

@@ -21,13 +21,6 @@
     "\n",
     "Registries can anchor dry & wetlab work to help find, access & model data. No more ambiguity about model dimensions and labels!\n",
     "\n",
-    "```\n",
-    "\n",
-    "```{toctree}\n",
-    ":hidden:\n",
-    ":maxdepth: 1\n",
-    "\n",
-    "../lnschema-bionty\n",
     "```"
    ]
   },
@@ -59,9 +52,7 @@
    "outputs": [],
    "source": [
     "import lamindb as ln\n",
-    "import lnschema_bionty as lb\n",
-    "\n",
-    "ln.settings.verbosity = 3  # show hints"
+    "import lnschema_bionty as lb"
    ]
   },
   {
@@ -330,7 +321,7 @@
    "source": [
     "You need to specify a field correspond to the values you are passing, for instance \"CellType.name\" or \"CellType.ontology_id\" in this case.\n",
     "\n",
-    "`ORM.from_values()` creates entries in the following steps:\n",
+    "`Registry.from_values()` creates entries in the following steps:\n",
     "\n",
     "1. If existing DB records that match the input field values, return records without creating new\n",
     "2. If input values matches synonyms associated with existing DB records, return records without creating new\n",