lamindb 1.2a2__tar.gz → 1.3.1__tar.gz

{lamindb-1.2a2 → lamindb-1.3.1}/.github/workflows/build.yml

@@ -68,8 +68,8 @@ jobs:
       - uses: actions/setup-python@v5
         with:
           python-version: |
-            ${{ matrix.group == 'storage' && '3.10' ||
-                '3.12'
+            ${{ github.ref == 'refs/heads/release' && '3.10' ||
+                '3.13'
                 }}
 
       - name: cache pre-commit
@@ -163,15 +163,23 @@ jobs:
       - run: nox -s "install_ci(group='docs')"
       - uses: actions/download-artifact@v4
       - run: nox -s docs
-      - uses: nwtgck/actions-netlify@v1.2
-        with:
-          publish-dir: "_build/html"
-          production-deploy: ${{ github.event_name == 'push' }}
-          github-token: ${{ secrets.GITHUB_TOKEN }}
-          enable-commit-comment: false
-        env:
-          NETLIFY_AUTH_TOKEN: ${{ secrets.NETLIFY_AUTH_TOKEN }}
-          NETLIFY_SITE_ID: ${{ secrets.NETLIFY_SITE_ID }}
+      - run: rm -r ./_build/html/.doctrees # do not want to deploy with cloudflare
+      - uses: cloudflare/wrangler-action@v3
+        id: cloudflare
+        with:
+          apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
+          accountId: 472bdad691b4483dea759eadb37110bd
+          command: pages deploy "_build/html" --project-name=lamindb
+          gitHubToken: ${{ secrets.GITHUB_TOKEN }}
+      - uses: edumserrano/find-create-or-update-comment@v2
+        if: github.event_name == 'pull_request'
+        with:
+          issue-number: ${{ github.event.pull_request.number }}
+          body-includes: "Deployment URL"
+          comment-author: "github-actions[bot]"
+          body: |
+            Deployment URL: ${{ steps.cloudflare.outputs.deployment-url }}
+          edit-mode: replace
 
       - uses: peter-evans/repository-dispatch@v2
         if: ${{ github.event_name == 'push' }}
@@ -187,7 +195,7 @@ jobs:
       - uses: actions/checkout@v4
       - uses: actions/setup-python@v5
         with:
-          python-version: "3.12"
+          python-version: "3.13"
       - run: |
           python -m pip install -U uv
           uv pip install --system coverage[toml]

{lamindb-1.2a2 → lamindb-1.3.1}/.gitignore

@@ -215,3 +215,6 @@ _build
 
 # VSCode
 .vscode
+
+# cxg
+!lamindb/curators/_cellxgene_schemas/schema_versions.csv

{lamindb-1.2a2 → lamindb-1.3.1}/CONTRIBUTING.md

@@ -15,10 +15,14 @@ For installation from GitHub, call:
 ```bash
 git clone --recursive https://github.com/laminlabs/lamindb
 pip install laminci
+python -m venv .venv
+source .venv/bin/activate
 nox -s install
 ```
 
-This will install a few dependencies from the git submodules linked [here](https://github.com/laminlabs/lamindb/tree/main/sub).
+This will install a few dependencies from the git submodules linked [here](https://github.com/laminlabs/lamindb/tree/main/sub), as well as packages
+like `pytest` and `pre-commit` that you'll need when developing.
+
 lamindb depends on several other packages that may require modifications for pull requests to successfully pass the continuous integration build.
 We suggest the following workflow if commits to any of the submodules are essential for the current modifications in lamindb:
 
@@ -78,6 +82,8 @@ and enabled for the repository via:
 gitmoji -i
 ```
 
+If you don't have `sudo` in your working environment, follow [these instructions](https://github.com/sindresorhus/guides/blob/main/npm-global-without-sudo.md).
+
 ## Documentation
 
 We build our documentation with an internal tool called `lndocs`.

{lamindb-1.2a2 → lamindb-1.3.1}/PKG-INFO

@@ -1,16 +1,17 @@
 Metadata-Version: 2.3
 Name: lamindb
-Version: 1.2a2
+Version: 1.3.1
 Summary: A data framework for biology.
 Author-email: Lamin Labs <open-source@lamin.ai>
-Requires-Python: >=3.10,<3.13
+Requires-Python: >=3.10,<3.14
 Description-Content-Type: text/markdown
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
-Requires-Dist: lamin_utils==0.13.10
-Requires-Dist: lamin_cli==1.1.0
-Requires-Dist: lamindb_setup[aws]==1.3.0
+Classifier: Programming Language :: Python :: 3.13
+Requires-Dist: lamin_utils==0.13.11
+Requires-Dist: lamin_cli==1.2.0
+Requires-Dist: lamindb_setup[aws]==1.4.0
 Requires-Dist: pyyaml
 Requires-Dist: pyarrow
 Requires-Dist: pandera
@@ -22,7 +23,7 @@ Requires-Dist: anndata>=0.8.0,<=0.11.3
 Requires-Dist: fsspec
 Requires-Dist: graphviz
 Requires-Dist: psycopg2-binary
-Requires-Dist: bionty ; extra == "bionty"
+Requires-Dist: bionty>=1.2a1 ; extra == "bionty"
 Requires-Dist: cellregistry ; extra == "cellregistry"
 Requires-Dist: clinicore ; extra == "clinicore"
 Requires-Dist: tomlkit ; extra == "dev"
@@ -39,7 +40,7 @@ Requires-Dist: faker-biology ; extra == "dev"
 Requires-Dist: django-schema-graph ; extra == "erdiagram"
 Requires-Dist: readfcs>=2.0.1 ; extra == "fcs"
 Requires-Dist: lamindb_setup[gcp] ; extra == "gcp"
-Requires-Dist: nbproject==0.10.5 ; extra == "jupyter"
+Requires-Dist: nbproject==0.10.6 ; extra == "jupyter"
 Requires-Dist: jupytext ; extra == "jupyter"
 Requires-Dist: nbconvert>=7.2.1 ; extra == "jupyter"
 Requires-Dist: mistune!=3.1.0 ; extra == "jupyter"

{lamindb-1.2a2 → lamindb-1.3.1}/docs/api.md

@@ -11,15 +11,6 @@ cli
 lamindb
 ```
 
-```{toctree}
-:maxdepth: 1
-:caption: R & REST
-:hidden:
-
-laminr
-rest
-```
-
 ```{toctree}
 :maxdepth: 1
 :caption: Modules
@@ -28,6 +19,4 @@ rest
 bionty
 wetlab
 clinicore
-cellregistry
-omop
 ```

lamindb-1.2a2/docs/curate-df.ipynb → lamindb-1.3.1/docs/curate.ipynb

@@ -5,13 +5,15 @@
    "id": "0",
    "metadata": {},
    "source": [
-    "# Curate DataFrames and AnnDatas\n",
+    "# Curate datasets\n",
     "\n",
     "Curating a dataset with LaminDB means three things:\n",
     "\n",
     "1. Validate that the dataset matches a desired schema\n",
     "2. In case the dataset doesn't validate, standardize it, e.g., by fixing typos or mapping synonyms\n",
-    "3. Annotate the dataset by linking it against metadata entities so that it becomes queryable"
+    "3. Annotate the dataset by linking it against metadata entities so that it becomes queryable\n",
+    "\n",
+    "In this guide we'll curate common data structures. Here is a [guide](/faq/curate-any) for the underlying low-level API."
    ]
   },
   {
@@ -61,23 +63,8 @@
     "import pandas as pd\n",
     "\n",
     "\n",
-    "df = pd.DataFrame(\n",
-    "    {\n",
-    "        \"perturbation\": pd.Categorical([\"DMSO\", \"IFNG\", \"DMSO\"]),\n",
-    "        \"temperature\": [37.2, 36.3, 38.2],\n",
-    "        \"cell_type\": pd.Categorical(\n",
-    "            [\n",
-    "                \"cerebral pyramidal neuron\",\n",
-    "                \"astrocytic glia\",\n",
-    "                \"oligodendrocyte\",\n",
-    "            ]\n",
-    "        ),\n",
-    "        \"assay_ontology_id\": pd.Categorical(\n",
-    "            [\"EFO:0008913\", \"EFO:0008913\", \"EFO:0008913\"]\n",
-    "        ),\n",
-    "        \"donor\": [\"D0001\", \"D0002\", None],\n",
-    "    },\n",
-    "    index=[\"obs1\", \"obs2\", \"obs3\"],\n",
+    "df = ln.core.datasets.small_dataset1(\n",
+    "    with_cell_type_synonym=True, with_cell_type_typo=True\n",
     ")\n",
     "df"
    ]
@@ -87,7 +74,7 @@
    "id": "5",
    "metadata": {},
    "source": [
-    "Define a schema to validate this dataset."
+    "Define a schema to define the minimal columns we expect in such a dataset."
    ]
   },
   {
@@ -102,15 +89,15 @@
    "outputs": [],
    "source": [
     "schema = ln.Schema(\n",
-    "    name=\"My example schema\",\n",
+    "    name=\"My immuno schema\",\n",
     "    features=[\n",
     "        ln.Feature(name=\"perturbation\", dtype=ln.ULabel).save(),\n",
-    "        ln.Feature(name=\"temperature\", dtype=float).save(),\n",
-    "        ln.Feature(name=\"cell_type\", dtype=bt.CellType).save(),\n",
-    "        ln.Feature(\n",
-    "            name=\"assay_ontology_id\", dtype=bt.ExperimentalFactor.ontology_id\n",
-    "        ).save(),\n",
+    "        ln.Feature(name=\"cell_type_by_model\", dtype=bt.CellType).save(),\n",
+    "        ln.Feature(name=\"cell_type_by_expert\", dtype=bt.CellType).save(),\n",
+    "        ln.Feature(name=\"assay_oid\", dtype=bt.ExperimentalFactor.ontology_id).save(),\n",
     "        ln.Feature(name=\"donor\", dtype=str, nullable=True).save(),\n",
+    "        ln.Feature(name=\"concentration\", dtype=str).save(),\n",
+    "        ln.Feature(name=\"treatment_time_h\", dtype=\"num\", coerce_dtype=True).save(),\n",
     "    ],\n",
     ").save()\n",
     "# display the associated features as a dataframe\n",
@@ -160,8 +147,8 @@
    "source": [
     "try:\n",
     "    curator.validate()\n",
-    "except ln.errors.ValidationError as error:\n",
-    "    print(error)"
+    "except ln.errors.ValidationError:\n",
+    "    pass"
    ]
   },
   {
@@ -193,10 +180,14 @@
    "cell_type": "code",
    "execution_count": null,
    "id": "13",
-   "metadata": {},
+   "metadata": {
+    "tags": [
+     "hide-output"
+    ]
+   },
    "outputs": [],
    "source": [
-    "curator.cat.standardize(\"cell_type\")"
+    "curator.cat.standardize(\"cell_type_by_expert\")"
    ]
   },
   {
@@ -251,8 +242,8 @@
    "outputs": [],
    "source": [
     "# here is an example for the \"cell_type\" column\n",
-    "cell_types = lookup[\"cell_type\"]\n",
-    "cell_types.cerebral_cortex_pyramidal_neuron"
+    "cell_types = lookup[\"cell_type_by_expert\"]\n",
+    "cell_types.cd8_positive_alpha_beta_t_cell"
    ]
   },
   {
@@ -266,9 +257,9 @@
    },
    "outputs": [],
    "source": [
-    "# fix the cell type\n",
-    "df.cell_type = df.cell_type.cat.rename_categories(\n",
-    "    {\"cerebral pyramidal neuron\": cell_types.cerebral_cortex_pyramidal_neuron.name}\n",
+    "# fix the cell type name\n",
+    "df[\"cell_type_by_expert\"] = df[\"cell_type_by_expert\"].cat.rename_categories(\n",
+    "    {\"CD8-pos alpha-beta T cell\": cell_types.cd8_positive_alpha_beta_t_cell.name}\n",
     ")"
    ]
   },
@@ -418,7 +409,11 @@
    "cell_type": "code",
    "execution_count": null,
    "id": "29",
-   "metadata": {},
+   "metadata": {
+    "tags": [
+     "hide-output"
+    ]
+   },
    "outputs": [],
    "source": [
     "anndata_schema.slots"
@@ -517,7 +512,11 @@
    "cell_type": "code",
    "execution_count": null,
    "id": "38",
-   "metadata": {},
+   "metadata": {
+    "tags": [
+     "hide-output"
+    ]
+   },
    "outputs": [],
    "source": [
     "artifact.features.slots"
@@ -558,7 +557,7 @@
    "id": "42",
    "metadata": {},
    "source": [
-    "If you need more control, you can access the underlying `\"var\"` and `\"obs\"` `DataFrameCurator` objects directly."
+    "If you need more control, you can access `DataFrameCurator` objects for the `\"var\"` and `\"obs\"` slots, respectively."
    ]
   },
   {
@@ -572,20 +571,23 @@
    },
    "outputs": [],
    "source": [
-    "curator.slots[\"var\"]\n",
-    "curator.slots[\"obs\"]"
+    "curator.slots"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
    "id": "44",
-   "metadata": {},
+   "metadata": {
+    "tags": [
+     "hide-output"
+    ]
+   },
    "outputs": [],
    "source": [
     "# revert the previous cell type standardization\n",
-    "df[\"cell_type\"] = df[\"cell_type\"].cat.rename_categories(\n",
-    "    {\"astrocyte\": \"astrocytic glia\"}\n",
+    "df[\"cell_type_by_expert\"] = df[\"cell_type_by_expert\"].cat.rename_categories(\n",
+    "    {\"B cell\": \"B-cell\"}\n",
     ")\n",
     "# an AnnData where a cell type matches a synonym\n",
     "adata_with_synonym = ad.AnnData(X=adata_validated.X, var=adata_validated.var, obs=df)\n",
@@ -596,31 +598,43 @@
    "cell_type": "code",
    "execution_count": null,
    "id": "45",
-   "metadata": {},
+   "metadata": {
+    "tags": [
+     "hide-output"
+    ]
+   },
    "outputs": [],
    "source": [
     "curator = ln.curators.AnnDataCurator(adata_with_synonym, anndata_schema)\n",
     "try:\n",
     "    curator.validate()\n",
-    "except ln.errors.ValidationError as error:\n",
-    "    print(error)"
+    "except ln.errors.ValidationError:\n",
+    "    pass"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
    "id": "46",
-   "metadata": {},
+   "metadata": {
+    "tags": [
+     "hide-output"
+    ]
+   },
    "outputs": [],
    "source": [
-    "curator.slots[\"obs\"].cat.standardize(\"cell_type\")"
+    "curator.slots[\"obs\"].cat.standardize(\"cell_type_by_expert\")"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
    "id": "47",
-   "metadata": {},
+   "metadata": {
+    "tags": [
+     "hide-output"
+    ]
+   },
    "outputs": [],
    "source": [
     "curator.validate()"
@@ -648,7 +662,7 @@
     "\n",
     "By following these steps, you can ensure your data is standardized and well-curated.\n",
     "\n",
-    "If you have datasets that aren't DataFrame-like or AnnData-like, read: {doc}`curate-any`."
+    "If you have other data structures, read: {doc}`/faq/curate-any`."
    ]
   },
   {