lamindb 0.73.2__tar.gz → 0.74.1__tar.gz

{lamindb-0.73.2 → lamindb-0.74.1}/.gitignore

@@ -60,6 +60,7 @@ test-search3
 test-search4
 test-search5
 default_storage
+test.ipynb
 
 # General
 .DS_Store
@@ -206,6 +207,7 @@ _build
 *.parquet
 *.bam
 *.fastq.gz
+*.pt
 
 # Pycharm
 .idea

{lamindb-0.73.2 → lamindb-0.74.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: lamindb
-Version: 0.73.2
+Version: 0.74.1
 Summary: A data framework for biology.
 Author-email: Lamin Labs <open-source@lamin.ai>
 Requires-Python: >=3.8
@@ -9,15 +9,15 @@ Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
-Requires-Dist: lnschema_core==0.69.0
-Requires-Dist: lamindb_setup==0.73.1
+Requires-Dist: lnschema_core==0.70.0
+Requires-Dist: lamindb_setup==0.73.2
 Requires-Dist: lamin_utils==0.13.2
 Requires-Dist: lamin_cli==0.14.0
 Requires-Dist: rapidfuzz
 Requires-Dist: pyarrow
 Requires-Dist: typing_extensions!=4.6.0
 Requires-Dist: python-dateutil
-Requires-Dist: anndata>=0.8.0,<0.10.8
+Requires-Dist: anndata>=0.8.0,<=0.10.8
 Requires-Dist: scipy<1.13.0rc1
 Requires-Dist: fsspec
 Requires-Dist: pandas
@@ -59,7 +59,7 @@ Provides-Extra: zarr
 
 # LaminDB - A data framework for biology
 
-- Manage storage & databases with a unified Python API ("lakehouse").
+- Manage data & metadata with a unified Python API ("lakehouse").
 - Track data lineage across notebooks & pipelines.
 - Integrate registries for experimental metadata & in-house ontologies.
 - Validate, standardize & annotate.

{lamindb-0.73.2 → lamindb-0.74.1}/README.md

@@ -4,7 +4,7 @@
 
 # LaminDB - A data framework for biology
 
-- Manage storage & databases with a unified Python API ("lakehouse").
+- Manage data & metadata with a unified Python API ("lakehouse").
 - Track data lineage across notebooks & pipelines.
 - Integrate registries for experimental metadata & in-house ontologies.
 - Validate, standardize & annotate.

{lamindb-0.73.2 → lamindb-0.74.1}/docs/annotate-for-developers.ipynb

@@ -24,6 +24,11 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
+    "Install the `lamindb` Python package:\n",
+    "```shell\n",
+    "pip install 'lamindb[jupyter,bionty]'\n",
+    "```\n",
+    "\n",
     "Let us create an instance that has {mod}`bionty` mounted:"
    ]
   },
@@ -61,7 +66,7 @@
     "bt.settings.auto_save_parents = False  # ignores ontological hierarchy\n",
     "ln.settings.verbosity = \"info\"\n",
     "ln.settings.transform.stem_uid = \"sU0y1kF3igep\"\n",
-    "ln.settings.transform.version = \"0\""
+    "ln.settings.transform.version = \"1\""
    ]
   },
   {

{lamindb-0.73.2 → lamindb-0.74.1}/docs/annotate.ipynb

@@ -29,6 +29,17 @@
     "```"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "c47f2b20",
+   "metadata": {},
+   "source": [
+    "Install the `lamindb` Python package:\n",
+    "```shell\n",
+    "pip install 'lamindb[bionty]'\n",
+    "```"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,

{lamindb-0.73.2 → lamindb-0.74.1}/docs/bio-registries.ipynb

@@ -43,6 +43,11 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
+    "Install the `lamindb` Python package:\n",
+    "```shell\n",
+    "pip install 'lamindb[bionty]'\n",
+    "```\n",
+    "\n",
     "Let us create an instance that has {mod}`bionty` mounted, which extends to include biological registries:"
    ]
   },

{lamindb-0.73.2 → lamindb-0.74.1}/docs/can-validate.ipynb

@@ -38,7 +38,12 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Setup"
+    "## Setup\n",
+    "\n",
+    "Install the `lamindb` Python package:\n",
+    "```shell\n",
+    "pip install 'lamindb[bionty]'\n",
+    "```"
    ]
   },
   {

{lamindb-0.73.2 → lamindb-0.74.1}/docs/changelog.md

@@ -12,6 +12,26 @@ LaminDB implements "migration-based versioning". When upgrading your LaminDB ins
 .. role:: small
 ```
 
+## 0.74
+
+### 0.74.1 {small}`2024-06-19`
+
+- ♻️ Refactor `ln.settings` [PR](https://github.com/laminlabs/lamindb/pull/1711) [@falexwolf](https://github.com/falexwolf)
+    - ✨ you can now pass custom names for scripts via `ln.settings.transform.name = "My script"`
+    - ⚠️ `ln.settings.storage` now returns a `StorageSettings` object (root via `ln.settings.storage.root`)
+- ✨ Support different join types in `QuerySet.df()` [PR](https://github.com/laminlabs/lamindb/pull/1709) [@insavchuk](https://github.com/insavchuk)
+- 📝 New [tiledbsoma guide](https://lamin.ai/docs/scrna6) [PR](https://github.com/laminlabs/lamin-usecases/pull/130) [Koncopd](https://github.com/Koncopd)
+- 📝 Update hub screenshots [PR](https://github.com/laminlabs/lamindb/pull/1714) [@sunnyosun](https://github.com/sunnyosun)
+
+### 0.74.0 {small}`2024-06-19`
+
+- ✨ You can now distinguish model-like and dataset-like artifacts via a `type` field in the `Artifact` registry
+    - 🚸 Leverage `artifact.params.add_values()` to annotate model-like artifacts like you leverage `artifact.features.add_values()` to annotate dataset-like artifacts
+    - 🏗️ Add `type` field to `Artifact`, allow linking model-like artifacts against params, validate params akin to validating features, enable features-based annotation with non-ulabels [PR](https://github.com/laminlabs/lamindb/pull/1690) [@falexwolf](https://github.com/falexwolf)
+    - 🚸 Support dict in `add_values` [PR](https://github.com/laminlabs/lamindb/pull/1705) [@Zethson](https://github.com/Zethson)
+- 📝 New [wandb guide](https://docs.lamin.ai/wandb) [PR](https://github.com/laminlabs/lamin-mlops/pull/2) [felix0097](https://github.com/felix0097)
+- ♻️ Refactor after upath upgrade [PR](https://github.com/laminlabs/lamindb/pull/1699) [PR](https://github.com/laminlabs/lamindb/pull/1700) [@Koncopd](https://github.com/Koncopd)
+
 ## 0.73
 
 ### 0.73.2 {small}`2024-06-13`

{lamindb-0.73.2 → lamindb-0.74.1}/docs/faq/acid.ipynb

@@ -26,6 +26,17 @@
     "## Setup"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "24f241b8",
+   "metadata": {},
+   "source": [
+    "Install the `lamindb` Python package:\n",
+    "```shell\n",
+    "pip install 'lamindb[jupyter,aws]'\n",
+    "```"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,

{lamindb-0.73.2 → lamindb-0.74.1}/docs/faq/delete.ipynb

@@ -21,6 +21,16 @@
     "## Setup"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Install the `lamindb` Python package:\n",
+    "```shell\n",
+    "pip install lamindb\n",
+    "```"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,

{lamindb-0.73.2 → lamindb-0.74.1}/docs/faq/idempotency.ipynb

@@ -23,21 +23,21 @@
     "\n",
     "### Metadata records\n",
     "\n",
-    "If you try to create any metadata record ({class}`~lamindb.core.Registry`) and {attr}`~lamindb.core.Settings.upon_create_search_names` is `True` (the default): \n",
+    "If you try to create any metadata record ({class}`~lamindb.core.Registry`) and {attr}`~lamindb.core.subsettings.CreationSettings.search_names` is `True` (the default): \n",
     "\n",
     "1. LaminDB will warn you if a record with similar `name` exists and display a table of similar existing records.\n",
     "2. You can then decide whether you'd like to save a record to the database or rather query an existing one from the table.\n",
     "3. If a name already has an exact match in a registry, LaminDB will return it instead of creating a new record. For versioned entities, also the version must be passed.\n",
     "\n",
-    "If you set {attr}`~lamindb.core.Settings.upon_create_search_names` to `False`, you'll directly populate the DB.\n",
+    "If you set {attr}`~lamindb.core.subsettings.CreationSettings.search_names` to `False`, you'll directly populate the DB.\n",
     "\n",
     "### Data: artifacts & collections\n",
     "\n",
-    "If you try to create a {class}`~lamindb.Artifact` object from the same content, depending on {attr}`~lamindb.core.Settings.upon_artifact_create_if_hash_exists`, \n",
+    "If you try to create a {class}`~lamindb.Artifact` object from the same content, depending on {attr}`~lamindb.core.subsettings.CreationSettings.artifact_if_hash_exists`, \n",
     "\n",
-    "- you'll get an existing object, if `upon_artifact_create_if_hash_exists = \"warn_return_existing\"` (the default)\n",
-    "- you'll get an error, if `upon_artifact_create_if_hash_exists = \"error\"`\n",
-    "- you'll get a warning and a new object, if `upon_artifact_create_if_hash_exists = \"warn_create_new\"`"
+    "- you'll get an existing object, if `creation.artifact_if_hash_exists = \"warn_return_existing\"` (the default)\n",
+    "- you'll get an error, if `creation.artifact_if_hash_exists = \"error\"`\n",
+    "- you'll get a warning and a new object, if `creation.artifact_if_hash_exists = \"warn_create_new\"`"
    ]
   },
   {
@@ -49,6 +49,17 @@
     "## Examples"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "076843ff",
+   "metadata": {},
+   "source": [
+    "Install the `lamindb` Python package:\n",
+    "```shell\n",
+    "pip install 'lamindb[jupyter]'\n",
+    "```"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -91,7 +102,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "assert ln.settings.upon_create_search_names"
+    "assert ln.settings.creation.search_names"
    ]
   },
   {
@@ -216,7 +227,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "ln.settings.upon_create_search_names = False"
+    "ln.settings.creation.search_names = False"
    ]
   },
   {
@@ -245,7 +256,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "ln.settings.upon_create_search_names = True"
+    "ln.settings.creation.search_names = True"
    ]
   },
   {
@@ -273,7 +284,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "assert ln.settings.upon_artifact_create_if_hash_exists == \"warn_return_existing\""
+    "assert ln.settings.creation.artifact_if_hash_exists == \"warn_return_existing\""
    ]
   },
   {
@@ -420,7 +431,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "ln.settings.upon_artifact_create_if_hash_exists = \"error\""
+    "ln.settings.creation.artifact_if_hash_exists = \"error\""
    ]
   },
   {
@@ -467,7 +478,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "ln.settings.upon_artifact_create_if_hash_exists = \"warn_create_new\""
+    "ln.settings.creation.artifact_if_hash_exists = \"warn_create_new\""
    ]
   },
   {
@@ -537,7 +548,11 @@
    "cell_type": "code",
    "execution_count": null,
    "id": "d2ca60bf",
-   "metadata": {},
+   "metadata": {
+    "tags": [
+     "hide-output"
+    ]
+   },
    "outputs": [],
    "source": [
     "!lamin delete --force test-idempotency\n",

{lamindb-0.73.2 → lamindb-0.74.1}/docs/faq/import-schema.ipynb

@@ -8,6 +8,16 @@
     "# What happens if I import a schema module without lamindb?"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Install the `lamindb` Python package:\n",
+    "```shell\n",
+    "pip install 'lamindb[bionty]'\n",
+    "```"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,

{lamindb-0.73.2 → lamindb-0.74.1}/docs/faq/keep-artifacts-local.ipynb

@@ -7,6 +7,16 @@
     "# Keep artifacts local in a cloud instance"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Install the `lamindb` Python package:\n",
+    "```shell\n",
+    "pip install 'lamindb[jupyter,aws]'\n",
+    "```"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -112,7 +122,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "storage_record = ln.Storage.filter(root=ln.settings.storage_local).one()\n",
+    "storage_record = ln.Storage.filter(root=ln.settings.storage_local.root).one()\n",
     "storage_record.description = \"Files stored locally in site X on server Y for reason ABC\"\n",
     "storage_record.save()\n",
     "ln.Storage.df()"
@@ -165,7 +175,7 @@
    },
    "outputs": [],
    "source": [
-    "ln.settings.storage_local.view_tree()"
+    "ln.settings.storage_local.root.view_tree()"
    ]
   },
   {
@@ -179,7 +189,7 @@
    "outputs": [],
    "source": [
     "assert local_path.exists()\n",
-    "assert artifact.path.as_posix().startswith(ln.setup.settings.instance.storage_local.root.as_posix())"
+    "assert artifact.path.as_posix().startswith(ln.settings.storage_local.root.as_posix())"
    ]
   },
   {

{lamindb-0.73.2 → lamindb-0.74.1}/docs/faq/key.ipynb

@@ -24,6 +24,16 @@
     "## Setup"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Install the `lamindb` Python package:\n",
+    "```shell\n",
+    "pip install 'lamindb[jupyter]'\n",
+    "```"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -171,7 +181,7 @@
    "metadata": {},
    "source": [
     "By default, Lamin uses virtual `keys` that are only reflected in the database but not in storage.\n",
-    "It is possible to turn this behavior off by setting `ln.settings.artifact_use_virtual_keys = False`.\n",
+    "It is possible to turn this behavior off by setting `ln.settings.creation._artifact_use_virtual_keys = False`.\n",
     "Generally, we discourage disabling this setting manually. For more details we refer to {doc}`docs:faq/storage`."
    ]
   },
@@ -181,7 +191,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "ln.settings.artifact_use_virtual_keys"
+    "ln.settings.creation._artifact_use_virtual_keys"
    ]
   },
   {
@@ -772,9 +782,9 @@
    "metadata": {},
    "source": [
     "LaminDB can ingest legacy data that already had a structure in their storage.\n",
-    "In such cases, it disables `artifact_use_virtual_keys` and the artifacts are ingested with their actual storage location.\n",
+    "In such cases, it disables `_artifact_use_virtual_keys` and the artifacts are ingested with their actual storage location.\n",
     "It might be therefore be possible that `Files` stored in different storage roots may be associated with a single `Collection`.\n",
-    "To simulate this, we are disabling `artifact_use_virtual_keys` and ingest artifacts stored in a different path (the \"legacy data\")."
+    "To simulate this, we are disabling `_artifact_use_virtual_keys` and ingest artifacts stored in a different path (the \"legacy data\")."
    ]
   },
   {
@@ -783,7 +793,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "ln.settings.artifact_use_virtual_keys = False"
+    "ln.settings.creation._artifact_use_virtual_keys = False"
    ]
   },
   {
@@ -869,7 +879,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "ln.settings.artifact_use_virtual_keys = True"
+    "ln.settings.creation._artifact_use_virtual_keys = True"
    ]
   },
   {

{lamindb-0.73.2 → lamindb-0.74.1}/docs/faq/reference-field.ipynb

@@ -16,6 +16,16 @@
     "We have `reference` and `reference_type` fields for this purpose, they are available for {class}`~lamindb.Collection`, {class}`~lamindb.Transform`, {class}`~lamindb.Run` and {class}`~lamindb.ULabel`."
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Install the `lamindb` Python package:\n",
+    "```shell\n",
+    "pip install lamindb\n",
+    "```"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,

{lamindb-0.73.2 → lamindb-0.74.1}/docs/faq/setup.ipynb

@@ -9,6 +9,17 @@
     "# What happens when importing lamindb and the instance is not yet setup?"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "a8a57840",
+   "metadata": {},
+   "source": [
+    "Install the `lamindb` Python package:\n",
+    "```shell\n",
+    "pip install 'lamindb[jupyter]'\n",
+    "```"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -54,7 +65,7 @@
    },
    "outputs": [],
    "source": [
-    "with pytest.raises(ln.InstanceNotSetupError):\n",
+    "with pytest.raises(ln._InstanceNotSetupError):\n",
     "    ln.track()"
    ]
   },

{lamindb-0.73.2 → lamindb-0.74.1}/docs/faq/track-run-inputs.ipynb

@@ -23,6 +23,16 @@
     "## Setup"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Install the `lamindb` Python package:\n",
+    "```shell\n",
+    "pip install 'lamindb[jupyter]'\n",
+    "```"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,

{lamindb-0.73.2 → lamindb-0.74.1}/docs/faq/visibility.ipynb

@@ -19,6 +19,16 @@
     "## Setup"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Install the `lamindb` Python package:\n",
+    "```shell\n",
+    "pip install lamindb\n",
+    "```"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,

{lamindb-0.73.2 → lamindb-0.74.1}/docs/introduction.ipynb

@@ -36,7 +36,7 @@
     ":::\n",
     "\n",
     "Basic features of LaminHub are free.\n",
-    "Enterprise features hosted in your or our infrastructure are available on a [paid plan](https://lamin.ai/pricing)!"
+    "Advanced features hosted in your or our infrastructure are available on a [paid plan](https://lamin.ai/pricing)."
    ]
   },
   {
@@ -990,18 +990,18 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "See an example for this introductory notebook [here](https://lamin.ai/laminlabs/lamindata/transform/FPnfDtJz8qbE5zKv).\n",
+    "See an example for this introductory notebook [here](https://lamin.ai/laminlabs/lamindata/transform/FPnfDtJz8qbE65cN/fMfi7IHYSylS5AY6GXtW).\n",
     "\n",
     ":::{dropdown} Show me a screenshot\n",
     "\n",
-    "<img src=\"https://lamin-site-assets.s3.amazonaws.com/.lamindb/RGXj5wcAf7EAc6J8aBoM.png\" width=\"700px\">\n",
+    "<img src=\"https://lamin-site-assets.s3.amazonaws.com/.lamindb/RGXj5wcAf7EAc6J8QksL.png\" width=\"700px\">\n",
     "\n",
     ":::\n",
     "\n",
     "If you want to cache a notebook or script, call:\n",
     "\n",
     "```bash\n",
-    "lamin get https://lamin.ai/laminlabs/lamindata/transform/FPnfDtJz8qbE5zKv\n",
+    "lamin get https://lamin.ai/laminlabs/lamindata/transform/FPnfDtJz8qbE65cN\n",
     "```\n"
    ]
   },
@@ -1121,19 +1121,28 @@
     "\n",
     "LaminDB and its plug-ins consist in open-source Python libraries & publicly hosted metadata assets:\n",
     "\n",
-    "- [lamindb](https://github.com/laminlabs/lamindb): Core API, which builds on the [core schema](https://github.com/laminlabs/lnschema-core).\n",
+    "- [lamindb](https://github.com/laminlabs/lamindb): Core package.\n",
     "- [bionty](https://github.com/laminlabs/bionty): Registries for basic biological entities, coupled to public ontologies.\n",
-    "- [wetlab](https://github.com/laminlabs/wetlab): An (exemplary) wetlab schema.\n",
-    "- [guides](https://github.com/laminlabs/lamindb/tree/main/docs/): Guides.\n",
+    "- [wetlab](https://github.com/laminlabs/wetlab): Default wetlab schema.\n",
+    "- [guides](https://github.com/laminlabs/lamindb/tree/main/docs): Guides.\n",
     "- [usecases](https://github.com/laminlabs/lamin-usecases): Use cases.\n",
     "\n",
-    "LaminHub is not open-sourced.\n",
+    "All immediate dependencies are available as git submodules [here](https://github.com/laminlabs/lamindb/tree/main/sub), for instance,\n",
+    "\n",
+    "- [lnschema-core](https://github.com/laminlabs/lnschema-core): Core schema.\n",
+    "- [lnschema-bionty](https://github.com/laminlabs/lnschema-core): Bionty schema.\n",
+    "- [lamindb-setup](https://github.com/laminlabs/lamindb-setup): Setup & configure LaminDB.\n",
+    "- [lamin-cli](https://github.com/laminlabs/lamin-cli): CLI for `lamindb` and `lamindb-setup`.\n",
+    "\n",
+    "For a comprehensive list of open-sourced software, browse our [GitHub account](https://github.com/laminlabs).\n",
     "\n",
-    "<!-- [lamindb-setup](https://github.com/laminlabs/lamindb-setup): Setup & configure LaminDB, client for LaminHub. -->\n",
-    "<!-- [lamin-cli](https://github.com/laminlabs/lamin-cli): CLI for `lamindb` and `lamindb-setup`. -->\n",
-    "<!-- [lamin-utils](https://github.com/laminlabs/lamin-utils): Generic utilities, e.g., a logger. -->\n",
-    "<!-- [readfcs](https://github.com/laminlabs/readfcs): FCS artifact reader. -->\n",
-    "<!-- [bionty-assets](https://github.com/laminlabs/bionty-assets): Hosted assets of parsed public biological ontologies. -->\n",
+    "- [lamin-utils](https://github.com/laminlabs/lamin-utils): Generic utilities, e.g., a logger.\n",
+    "- [readfcs](https://github.com/laminlabs/readfcs): FCS artifact reader.\n",
+    "- [nbproject](https://github.com/laminlabs/readfcs): Light-weight Jupyter notebook tracker.\n",
+    "- [bionty-base](https://github.com/laminlabs/bionty-assets): Base API for public biological registries.\n",
+    "- [bionty-assets](https://github.com/laminlabs/bionty-assets): Assets for public biological ontologies.\n",
+    "\n",
+    "LaminHub is not open-sourced.\n",
     "\n",
     "### Influences\n",
     "\n",