lightly-studio 0.3.3__tar.gz → 0.3.4__tar.gz

{lightly_studio-0.3.3 → lightly_studio-0.3.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: lightly-studio
-Version: 0.3.3
+Version: 0.3.4
 Summary: LightlyStudio is a lightweight, fast, and easy-to-use data exploration tool for data scientists and engineers.
 Classifier: Operating System :: MacOS :: MacOS X
 Classifier: Operating System :: Microsoft :: Windows
@@ -10,7 +10,8 @@ Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
-Requires-Python: >=3.8
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: <3.13,>=3.8
 Requires-Dist: annotated-types==0.7.0
 Requires-Dist: duckdb-engine<0.17,>=0.15.0
 Requires-Dist: duckdb<1.3,>=1.2.2
@@ -60,7 +61,7 @@ Description-Content-Type: text/markdown
 
 We at **[Lightly](https://lightly.ai)** created **LightlyStudio**, an open-source tool
 designed to supercharge your data curation workflows for computer vision datasets. Explore
-your data, visualize annotations and crops, tag samples, and export curated lists to improve
+your data, visualize captions, annotations and crops, tag samples, and export curated lists to improve
 your machine learning pipelines. And much more!
 
 LightlyStudio runs entirely locally on your machine, keeping your data private. It consists
@@ -72,7 +73,7 @@ Using LightlyStudio typically involves these steps:
 
 1.  **Index Your Dataset:** Run a Python script using the `lightly_studio` library to process your local dataset (images and annotations) and save metadata into a local `lightly_studio.db` file.
 2.  **Launch the UI:** The script then starts a local web server.
-3.  **Explore & Curate:** Use the UI to visualize images, annotations, and object crops. Filter and search your data (experimental text search available). Apply tags to interesting samples (e.g., "mislabeled", "review").
+3.  **Explore & Curate:** Use the UI to visualize images, annotations, captions, and object crops. Filter and search your data (experimental text search available). Apply tags to interesting samples (e.g., "mislabeled", "review").
 4.  **Export Curated Data:** Export information (like filenames) for your tagged samples from the UI to use downstream.
 5.  **Stop the Server:** Close the terminal running the script (Ctrl+C) when done.
 
@@ -111,7 +112,7 @@ The library is OS-independent and works on Windows, Linux, and macOS.
 
 ```shell
 # 1. Create and activate a virtual environment (Recommended)
-# On Linux/macOS:
+# On Linux/MacOS:
 python3 -m venv venv
 source venv/bin/activate
 
@@ -139,14 +140,12 @@ following contents in the same directory that contains the `dataset_examples/` f
 ```python
 # example_yolo.py
 
-from pathlib import Path
 import lightly_studio as ls
 
 # Create a dataset and add the samples from the yolo format
 dataset = ls.Dataset.create()
 dataset.add_samples_from_yolo(
     data_yaml="dataset_examples/road_signs_yolo/data.yaml",
-    input_split="test",
 )
 
 # Start the UI application on the port 8001.
@@ -202,7 +201,6 @@ the `dataset_examples/` folder:
 ```python
 # example_coco.py
 
-from pathlib import Path
 import lightly_studio as ls
 
 # Create a dataset and add the samples from the coco format
@@ -245,6 +243,56 @@ COCO uses a single JSON file containing all annotations. The format consists of
 
 </details>
 
+### COCO Captions
+
+To run a caption example using a COCO dataset, create a file named
+`example_coco_captions.py` with the following contents in the same directory that contains
+the `dataset_examples/` folder:
+
+```python
+# example_coco_captions.py
+
+import lightly_studio as ls
+
+# Create a dataset and add the samples from the coco format
+dataset = ls.Dataset.create()
+dataset.add_samples_from_coco_caption(
+    annotations_json="dataset_examples/coco_subset_128_images/captions_train2017.json",
+    images_path="dataset_examples/coco_subset_128_images/images",
+)
+
+# Start the UI application on the port 8001.
+ls.start_gui()
+```
+
+Run the script:
+
+```
+python example_coco_captions.py
+```
+
+Now you can inspect samples with their assigned captions in the app. When you are done,
+stop the app by pressing Ctrl+C in the terminal.
+
+<details>
+<summary>The COCO format details:</summary>
+
+```
+coco_subset_128_images/
+├── images/
+│   ├── image1.jpg
+│   ├── image2.jpg
+│   └── ...
+└── captions_train2017.json        # Single JSON file containing all captions
+```
+
+COCO uses a single JSON file containing all captions. The format consists of three main components:
+
+- Images: Defines metadata for each image in the dataset.
+- Annotations: Defines the captions.
+
+</details>
+
 ## 🔍 How It Works
 
 1.  Your **Python script** uses the `lightly_studio` **Dataset**.
@@ -330,7 +378,6 @@ import lightly_studio as ls
 dataset = ls.Dataset.create()
 dataset.add_samples_from_yolo(
     data_yaml="my_yolo_dataset/data.yaml",
-    input_split="val",
 )
 
 ls.start_gui()
@@ -628,12 +675,12 @@ query.add_tag("tag_name")
 
 #### Export Samples
 
-Currently, exporting to the COCO object detection format is supported. The following example shows how to export the samples in the query to a COCO JSON file:
+Currently, exporting to the COCO object detection format is supported and only annotations
+of type object detection are exported. The following example exports the samples in the query
+to a COCO JSON file named `coco_export.json`:
 
 ```py
-from pathlib import Path
-
-query.export().to_coco_object_detections(Path("coco_export.json"))
+query.export().to_coco_object_detections()
 ```
 
 ### Examples
@@ -697,7 +744,11 @@ samples.
 Set the `LIGHTLY_STUDIO_LICENSE_KEY` environment variable before using selection features:
 
 ```bash
+# On Linux/MacOS
 export LIGHTLY_STUDIO_LICENSE_KEY="license_key_here"
+
+# On Windows (PowerShell)
+$env:LIGHTLY_STUDIO_LICENSE_KEY="license_key_here"
 ```
 
 Alternatively, set it inside your Python script:
@@ -707,6 +758,12 @@ import os
 os.environ["LIGHTLY_STUDIO_LICENSE_KEY"] = "license_key_here"
 ```
 
+Or in a `.env` file:
+
+```
+LIGHTLY_STUDIO_LICENSE_KEY="license_key_here"
+```
+
 #### Diversity Selection
 
 Diversity selection can be configured directly from a `DatasetQuery`. The example below showcases a simple case of selecting diverse samples.
@@ -796,19 +853,27 @@ with open("export.txt", "w") as f:
 
 ## 📚 **FAQ**
 
-### Are the datasets persistent?
+### Does LightlyStudio persist the datasets?
 
-Yes, the information about datasets is persistent and stored in the db file. You can see it after the dataset is processed.
-If you rerun the loader it will create a new dataset representing the same dataset, keeping the previous dataset information untouched.
+Yes, the information about datasets is persisted in a database file. You can see inspect
+it after the dataset is processed. Use `Dataset.load()` to load a dataset from a pre-existing
+database.
 
 ### Can I change the database path?
 
-Not yet. The database is stored in the working directory by default.
+Yes, the database can be selected as follows:
+```py
+import lightly_studio as ls
+
+ls.db_manager.connect(db_file="custom.db")
+```
 
-### Can I launch in another Python script or do I have to do it in the same script?
+### Can I use LightlyStudio from two scripts in parallel?
 
-It is possible to use only one script at the same time because we lock the db file for the duration of the script.
+Only one script can be run at one time as the app uses a database lock for data integrity.
 
-### Can I change the API backend port?
+### Can I change the API backend host and port?
 
-Yes. To change the port set the LIGHTLY_STUDIO_PORT variable to your preffered value. If at runtime the port is unavailable it will try to set it to a random value.
+Yes, by setting environment variables. For the host set the LIGHTLY_STUDIO_HOST variable,
+to change the port set the LIGHTLY_STUDIO_PORT variable. Note that if the port is unavailable
+at runtime the app uses a random port number.

{lightly_studio-0.3.3 → lightly_studio-0.3.4}/USAGE.md

@@ -19,7 +19,7 @@
 
 We at **[Lightly](https://lightly.ai)** created **LightlyStudio**, an open-source tool
 designed to supercharge your data curation workflows for computer vision datasets. Explore
-your data, visualize annotations and crops, tag samples, and export curated lists to improve
+your data, visualize captions, annotations and crops, tag samples, and export curated lists to improve
 your machine learning pipelines. And much more!
 
 LightlyStudio runs entirely locally on your machine, keeping your data private. It consists
@@ -31,7 +31,7 @@ Using LightlyStudio typically involves these steps:
 
 1.  **Index Your Dataset:** Run a Python script using the `lightly_studio` library to process your local dataset (images and annotations) and save metadata into a local `lightly_studio.db` file.
 2.  **Launch the UI:** The script then starts a local web server.
-3.  **Explore & Curate:** Use the UI to visualize images, annotations, and object crops. Filter and search your data (experimental text search available). Apply tags to interesting samples (e.g., "mislabeled", "review").
+3.  **Explore & Curate:** Use the UI to visualize images, annotations, captions, and object crops. Filter and search your data (experimental text search available). Apply tags to interesting samples (e.g., "mislabeled", "review").
 4.  **Export Curated Data:** Export information (like filenames) for your tagged samples from the UI to use downstream.
 5.  **Stop the Server:** Close the terminal running the script (Ctrl+C) when done.
 
@@ -70,7 +70,7 @@ The library is OS-independent and works on Windows, Linux, and macOS.
 
 ```shell
 # 1. Create and activate a virtual environment (Recommended)
-# On Linux/macOS:
+# On Linux/MacOS:
 python3 -m venv venv
 source venv/bin/activate
 
@@ -98,14 +98,12 @@ following contents in the same directory that contains the `dataset_examples/` f
 ```python
 # example_yolo.py
 
-from pathlib import Path
 import lightly_studio as ls
 
 # Create a dataset and add the samples from the yolo format
 dataset = ls.Dataset.create()
 dataset.add_samples_from_yolo(
     data_yaml="dataset_examples/road_signs_yolo/data.yaml",
-    input_split="test",
 )
 
 # Start the UI application on the port 8001.
@@ -161,7 +159,6 @@ the `dataset_examples/` folder:
 ```python
 # example_coco.py
 
-from pathlib import Path
 import lightly_studio as ls
 
 # Create a dataset and add the samples from the coco format
@@ -204,6 +201,56 @@ COCO uses a single JSON file containing all annotations. The format consists of
 
 </details>
 
+### COCO Captions
+
+To run a caption example using a COCO dataset, create a file named
+`example_coco_captions.py` with the following contents in the same directory that contains
+the `dataset_examples/` folder:
+
+```python
+# example_coco_captions.py
+
+import lightly_studio as ls
+
+# Create a dataset and add the samples from the coco format
+dataset = ls.Dataset.create()
+dataset.add_samples_from_coco_caption(
+    annotations_json="dataset_examples/coco_subset_128_images/captions_train2017.json",
+    images_path="dataset_examples/coco_subset_128_images/images",
+)
+
+# Start the UI application on the port 8001.
+ls.start_gui()
+```
+
+Run the script:
+
+```
+python example_coco_captions.py
+```
+
+Now you can inspect samples with their assigned captions in the app. When you are done,
+stop the app by pressing Ctrl+C in the terminal.
+
+<details>
+<summary>The COCO format details:</summary>
+
+```
+coco_subset_128_images/
+├── images/
+│   ├── image1.jpg
+│   ├── image2.jpg
+│   └── ...
+└── captions_train2017.json        # Single JSON file containing all captions
+```
+
+COCO uses a single JSON file containing all captions. The format consists of three main components:
+
+- Images: Defines metadata for each image in the dataset.
+- Annotations: Defines the captions.
+
+</details>
+
 ## 🔍 How It Works
 
 1.  Your **Python script** uses the `lightly_studio` **Dataset**.
@@ -289,7 +336,6 @@ import lightly_studio as ls
 dataset = ls.Dataset.create()
 dataset.add_samples_from_yolo(
     data_yaml="my_yolo_dataset/data.yaml",
-    input_split="val",
 )
 
 ls.start_gui()
@@ -587,12 +633,12 @@ query.add_tag("tag_name")
 
 #### Export Samples
 
-Currently, exporting to the COCO object detection format is supported. The following example shows how to export the samples in the query to a COCO JSON file:
+Currently, exporting to the COCO object detection format is supported and only annotations
+of type object detection are exported. The following example exports the samples in the query
+to a COCO JSON file named `coco_export.json`:
 
 ```py
-from pathlib import Path
-
-query.export().to_coco_object_detections(Path("coco_export.json"))
+query.export().to_coco_object_detections()
 ```
 
 ### Examples
@@ -656,7 +702,11 @@ samples.
 Set the `LIGHTLY_STUDIO_LICENSE_KEY` environment variable before using selection features:
 
 ```bash
+# On Linux/MacOS
 export LIGHTLY_STUDIO_LICENSE_KEY="license_key_here"
+
+# On Windows (PowerShell)
+$env:LIGHTLY_STUDIO_LICENSE_KEY="license_key_here"
 ```
 
 Alternatively, set it inside your Python script:
@@ -666,6 +716,12 @@ import os
 os.environ["LIGHTLY_STUDIO_LICENSE_KEY"] = "license_key_here"
 ```
 
+Or in a `.env` file:
+
+```
+LIGHTLY_STUDIO_LICENSE_KEY="license_key_here"
+```
+
 #### Diversity Selection
 
 Diversity selection can be configured directly from a `DatasetQuery`. The example below showcases a simple case of selecting diverse samples.
@@ -755,19 +811,27 @@ with open("export.txt", "w") as f:
 
 ## 📚 **FAQ**
 
-### Are the datasets persistent?
+### Does LightlyStudio persist the datasets?
 
-Yes, the information about datasets is persistent and stored in the db file. You can see it after the dataset is processed.
-If you rerun the loader it will create a new dataset representing the same dataset, keeping the previous dataset information untouched.
+Yes, the information about datasets is persisted in a database file. You can see inspect
+it after the dataset is processed. Use `Dataset.load()` to load a dataset from a pre-existing
+database.
 
 ### Can I change the database path?
 
-Not yet. The database is stored in the working directory by default.
+Yes, the database can be selected as follows:
+```py
+import lightly_studio as ls
+
+ls.db_manager.connect(db_file="custom.db")
+```
 
-### Can I launch in another Python script or do I have to do it in the same script?
+### Can I use LightlyStudio from two scripts in parallel?
 
-It is possible to use only one script at the same time because we lock the db file for the duration of the script.
+Only one script can be run at one time as the app uses a database lock for data integrity.
 
-### Can I change the API backend port?
+### Can I change the API backend host and port?
 
-Yes. To change the port set the LIGHTLY_STUDIO_PORT variable to your preffered value. If at runtime the port is unavailable it will try to set it to a random value.
+Yes, by setting environment variables. For the host set the LIGHTLY_STUDIO_HOST variable,
+to change the port set the LIGHTLY_STUDIO_PORT variable. Note that if the port is unavailable
+at runtime the app uses a random port number.

{lightly_studio-0.3.3 → lightly_studio-0.3.4}/pyproject.toml

@@ -1,15 +1,16 @@
 [project]
 name = "lightly-studio"
-version = "0.3.3"
+version = "0.3.4"
 description = "LightlyStudio is a lightweight, fast, and easy-to-use data exploration tool for data scientists and engineers."
 readme = "USAGE.md"
-requires-python = ">=3.8"
+requires-python = ">=3.8,<3.13"
 classifiers = [
     "Programming Language :: Python :: 3",
     "Programming Language :: Python :: 3.8",
     "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
     "Operating System :: POSIX :: Linux",
     "Operating System :: MacOS :: MacOS X",
     "Operating System :: Microsoft :: Windows",

{lightly_studio-0.3.3 → lightly_studio-0.3.4}/src/lightly_studio/api/app.py

@@ -16,6 +16,7 @@ from lightly_studio.api.routes import healthz, images, webapp
 from lightly_studio.api.routes.api import (
     annotation,
     annotation_label,
+    caption,
     classifier,
     dataset,
     dataset_tag,
@@ -89,6 +90,7 @@ api_router.include_router(export.export_router)
 api_router.include_router(sample.samples_router)
 api_router.include_router(annotation_label.annotations_label_router)
 api_router.include_router(annotation.annotations_router)
+api_router.include_router(caption.captions_router)
 api_router.include_router(text_embedding.text_embedding_router)
 api_router.include_router(settings.settings_router)
 api_router.include_router(classifier.classifier_router)

lightly_studio-0.3.4/src/lightly_studio/api/routes/api/caption.py

@@ -0,0 +1,30 @@
+"""API routes for dataset captions."""
+
+from __future__ import annotations
+
+from uuid import UUID
+
+from fastapi import APIRouter, Depends, Path
+from typing_extensions import Annotated
+
+from lightly_studio.api.routes.api.validators import Paginated, PaginatedWithCursor
+from lightly_studio.db_manager import SessionDep
+from lightly_studio.models.caption import CaptionsListView
+from lightly_studio.resolvers import caption_resolver
+from lightly_studio.resolvers.caption_resolver import GetAllCaptionsResult
+
+captions_router = APIRouter(prefix="/datasets/{dataset_id}", tags=["captions"])
+
+
+@captions_router.get("/captions", response_model=CaptionsListView)
+def read_captions(
+    dataset_id: Annotated[UUID, Path(title="Dataset Id")],
+    session: SessionDep,
+    pagination: Annotated[PaginatedWithCursor, Depends()],
+) -> GetAllCaptionsResult:
+    """Retrieve captions for a dataset."""
+    return caption_resolver.get_all(
+        session=session,
+        dataset_id=dataset_id,
+        pagination=Paginated(offset=pagination.offset, limit=pagination.limit),
+    )

{lightly_studio-0.3.3 → lightly_studio-0.3.4}/src/lightly_studio/api/routes/api/embeddings2d.py

@@ -3,25 +3,40 @@
 from __future__ import annotations
 
 import io
+from uuid import UUID
 
 import numpy as np
 import pyarrow as pa
 from fastapi import APIRouter, HTTPException, Response
 from numpy.typing import NDArray
 from pyarrow import ipc
+from pydantic import BaseModel, Field
 from sklearn.manifold import TSNE
 from sqlmodel import select
 
 from lightly_studio.db_manager import SessionDep
 from lightly_studio.models.dataset import DatasetTable
 from lightly_studio.models.embedding_model import EmbeddingModelTable
-from lightly_studio.resolvers import sample_embedding_resolver
+from lightly_studio.resolvers import sample_embedding_resolver, sample_resolver
+from lightly_studio.resolvers.samples_filter import SampleFilter
 
 embeddings2d_router = APIRouter()
 
 
-@embeddings2d_router.get("/embeddings2d/tsne")
-def get_embeddings2d__tsne(session: SessionDep) -> Response:
+class GetEmbeddings2DRequest(BaseModel):
+    """Request body for retrieving 2D embeddings."""
+
+    filters: SampleFilter | None = Field(
+        None,
+        description="Filter parameters identifying matching samples",
+    )
+
+
+@embeddings2d_router.post("/embeddings2d/tsne")
+def get_embeddings2d__tsne(
+    session: SessionDep,
+    body: GetEmbeddings2DRequest | None = None,
+) -> Response:
     """Return 2D embeddings serialized as an Arrow stream."""
     # TODO(Malte, 09/2025): Support choosing the dataset via API parameter.
     dataset = session.exec(select(DatasetTable).limit(1)).first()
@@ -37,7 +52,6 @@ def get_embeddings2d__tsne(session: SessionDep) -> Response:
     if embedding_model is None:
         raise HTTPException(status_code=404, detail="No embedding model configured.")
 
-    # TODO(Malte, 09/2025): Support choosing a subset of samples via API parameter.
     embeddings = sample_embedding_resolver.get_all_by_dataset_id(
         session=session,
         dataset_id=dataset.dataset_id,
@@ -49,6 +63,22 @@ def get_embeddings2d__tsne(session: SessionDep) -> Response:
     x = embedding_values_tsne[:, 0]
     y = embedding_values_tsne[:, 1]
 
+    matching_sample_ids: set[UUID] | None = None
+    filters = body.filters if body else None
+    if filters:
+        matching_samples_result = sample_resolver.get_all_by_dataset_id(
+            session=session,
+            dataset_id=dataset.dataset_id,
+            filters=filters,
+        )
+        matching_sample_ids = {sample.sample_id for sample in matching_samples_result.samples}
+
+    sample_ids = [embedding.sample_id for embedding in embeddings]
+    if matching_sample_ids is None:
+        fulfils_filter = [1] * len(sample_ids)
+    else:
+        fulfils_filter = [1 if sample_id in matching_sample_ids else 0 for sample_id in sample_ids]
+
     # TODO(Malte, 09/2025): Save the 2D-embeddings in the database to avoid recomputing
     # them on every request.
 
@@ -57,6 +87,8 @@ def get_embeddings2d__tsne(session: SessionDep) -> Response:
         {
             "x": pa.array(x, type=pa.float32()),
             "y": pa.array(y, type=pa.float32()),
+            "fulfils_filter": pa.array(fulfils_filter, type=pa.uint8()),
+            "sample_id": pa.array([str(sample_id) for sample_id in sample_ids], type=pa.string()),
         }
     )
 

lightly_studio-0.3.4/src/lightly_studio/api/routes/api/metadata.py

@@ -0,0 +1,91 @@
+"""This module contains the API routes for managing datasets."""
+
+from __future__ import annotations
+
+from typing import List
+from uuid import UUID
+
+from fastapi import APIRouter, Depends, Path
+from pydantic import BaseModel, Field
+from typing_extensions import Annotated
+
+from lightly_studio.api.routes.api.dataset import get_and_validate_dataset_id
+from lightly_studio.db_manager import SessionDep
+from lightly_studio.metadata import compute_typicality
+from lightly_studio.models.dataset import DatasetTable
+from lightly_studio.models.metadata import MetadataInfoView
+from lightly_studio.resolvers import embedding_model_resolver
+from lightly_studio.resolvers.metadata_resolver.sample.get_metadata_info import (
+    get_all_metadata_keys_and_schema,
+)
+
+metadata_router = APIRouter(prefix="/datasets/{dataset_id}", tags=["metadata"])
+
+
+@metadata_router.get("/metadata/info", response_model=List[MetadataInfoView])
+def get_metadata_info(
+    session: SessionDep,
+    dataset_id: Annotated[UUID, Path(title="Dataset Id")],
+) -> list[MetadataInfoView]:
+    """Get all metadata keys and their schema for a dataset.
+
+    Args:
+        session: The database session.
+        dataset_id: The ID of the dataset.
+
+    Returns:
+        List of metadata info objects with name, type, and optionally min/max values
+        for numerical metadata types.
+    """
+    return get_all_metadata_keys_and_schema(session=session, dataset_id=dataset_id)
+
+
+class ComputeTypicalityRequest(BaseModel):
+    """Request model for computing typicality metadata."""
+
+    embedding_model_name: str | None = Field(
+        default=None,
+        description="Embedding model name (uses default if not specified)",
+    )
+    metadata_name: str = Field(
+        default="typicality",
+        description="Metadata field name (defaults to 'typicality')",
+    )
+
+
+@metadata_router.post(
+    "/metadata/typicality",
+    status_code=204,
+    response_model=None,
+)
+def compute_typicality_metadata(
+    session: SessionDep,
+    dataset: Annotated[
+        DatasetTable,
+        Depends(get_and_validate_dataset_id),
+    ],
+    request: ComputeTypicalityRequest,
+) -> None:
+    """Compute typicality metadata for a dataset.
+
+    Args:
+        session: The database session.
+        dataset: The dataset to compute typicality for.
+        request: Request parameters including optional embedding model name
+            and metadata field name.
+
+    Returns:
+        None (204 No Content on success).
+    """
+    embedding_model = embedding_model_resolver.get_by_name(
+        session=session,
+        dataset_id=dataset.dataset_id,
+        embedding_model_name=request.embedding_model_name,
+    )
+
+    compute_typicality.compute_typicality_metadata(
+        session=session,
+        dataset_id=dataset.dataset_id,
+        embedding_model_id=embedding_model.embedding_model_id,
+        metadata_name=request.metadata_name,
+    )