hirundo 0.1.16__tar.gz → 0.1.18__tar.gz

{hirundo-0.1.16 → hirundo-0.1.18}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hirundo
-Version: 0.1.16
+Version: 0.1.18
 Summary: This package is used to interface with Hirundo's platform. It provides a simple API to optimize your ML datasets.
 Author-email: Hirundo <dev@hirundo.io>
 License: MIT License
@@ -55,7 +55,7 @@ Requires-Dist: platformdirs>=4.3.6; extra == "dev"
 Requires-Dist: safety>=3.2.13; extra == "dev"
 Provides-Extra: docs
 Requires-Dist: sphinx>=7.4.7; extra == "docs"
-Requires-Dist: sphinx-autobuild>=2024.4.16; extra == "docs"
+Requires-Dist: sphinx-autobuild>=2024.9.3; extra == "docs"
 Requires-Dist: sphinx-click>=5.0.1; extra == "docs"
 Requires-Dist: autodoc_pydantic>=2.2.0; extra == "docs"
 Requires-Dist: furo; extra == "docs"
@@ -64,7 +64,7 @@ Requires-Dist: esbonio; extra == "docs"
 Requires-Dist: starlette>0.40.0; extra == "docs"
 Requires-Dist: markupsafe>=3.0.2; extra == "docs"
 Provides-Extra: pandas
-Requires-Dist: pandas>=2.2.2; extra == "pandas"
+Requires-Dist: pandas>=2.2.3; extra == "pandas"
 Provides-Extra: polars
 Requires-Dist: polars>=1.0.0; extra == "polars"
 Dynamic: license-file
@@ -75,40 +75,62 @@ This package exposes access to Hirundo APIs for dataset optimization for Machine
 
 Dataset optimization is currently available for datasets labelled for classification and object detection.
 
-
 Support dataset storage configs include:
-   - Google Cloud (GCP) Storage
-   - Amazon Web Services (AWS) S3
-   - Git LFS (Large File Storage) repositories (e.g. GitHub or HuggingFace)
+
+- Google Cloud (GCP) Storage
+- Amazon Web Services (AWS) S3
+- Git LFS (Large File Storage) repositories (e.g. GitHub or HuggingFace)
+
+Note: This Python package must be used alongside a Hirundo server, either the SaaS platform, a custom VPC deployment or an on-premises installation.
 
 Optimizing a classification dataset
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Currently ``hirundo`` requires a CSV file with the following columns (all columns are required):
-   - ``image_path``: The location of the image within the dataset ``root``
-   - ``label``: The label of the image, i.e. which the class that was annotated for this image
+Currently `hirundo` requires a CSV file with the following columns (all columns are required):
+
+- `image_path`: The location of the image within the dataset `data_root_url`
+- `class_name`: The semantic label, i.e. the class name of the class that the image was annotated as belonging to
+
+And outputs two Pandas DataFrames with the dataset columns as well as:
+
+Suspect DataFrame (filename: `mislabel_suspects.csv`) columns:
 
-And outputs a CSV with the same columns and:
-   - ``suspect_level``: mislabel suspect level
-   - ``suggested_label``: suggested label
-   - ``suggested_label_conf``: suggested label confidence
+- ``suspect_score``: mislabel suspect score
+- ``suspect_level``: mislabel suspect level
+- ``suspect_rank``: mislabel suspect ranking
+- ``suggested_class_name``: suggested semantic label
+- ``suggested_class_conf``: suggested semantic label confidence
+
+Errors and warnings DataFrame (filename: `invalid_data.csv`) columns:
+
+   - ``status``: status message (one of ``NO_LABELS`` / ``MISSING_IMAGE`` / ``INVALID_IMAGE``)
 
 Optimizing an object detection (OD) dataset
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Currently ``hirundo`` requires a CSV file with the following columns (all columns are required):
-   - ``image_path``: The location of the image within the dataset ``root``
-   - ``bbox_id``: The index of the bounding box within the dataset. Used to indicate label suspects
-   - ``label``: The label of the image, i.e. which the class that was annotated for this image
-   - ``x1``, ``y1``, ``x2``, ``y2``: The bounding box coordinates of the object within the image
 
-And outputs a CSV with the same columns and:
-   - ``suspect_level``: object mislabel suspect level
-   - ``suggested_label``: suggested object label
-   - ``suggested_label_conf``: suggested object label confidence
+- ``image_path``: The location of the image within the dataset ``data_root_url``
+- ``object_id``: The ID of the bounding box within the dataset. Used to indicate object suspects
+- ``class_name``: Object semantic label, i.e. the class name of the object that was annotated
+- ``xmin``: leftmost horizontal pixel coordinate of the object's bounding box
+- ``ymin``: uppermost vertical pixel coordinate of the object's bounding box
+- ``xmax``: rightmost horizontal pixel coordinate of the object's bounding box
+- ``ymax``: lowermost vertical pixel coordinate of the object's bounding box
 
-Note: This Python package must be used alongside a Hirundo server, either the SaaS platform, a custom VPC deployment or an on-premises installation.
 
+And outputs two Pandas DataFrames with the dataset columns as well as:
+
+Suspect DataFrame (filename: `mislabel_suspects.csv`) columns:
+
+- ``suspect_score``: object mislabel suspect score
+- ``suspect_level``: object mislabel suspect level
+- ``suspect_rank``: object mislabel suspect ranking
+- ``suggested_class_name``: suggested object semantic label
+- ``suggested_class_conf``: suggested object semantic label confidence
+
+Errors and warnings DataFrame (filename: `invalid_data.csv`) columns:
+   - ``status``: status message (one of ``NO_LABELS`` / ``MISSING_IMAGE`` / ``INVALID_IMAGE`` / ``INVALID_BBOX`` / ``INVALID_BBOX_SIZE``)
 
 ## Installation
 
@@ -117,6 +139,7 @@ You can install the codebase with a simple `pip install hirundo` to install the
 ## Usage
 
 Classification example:
+
 ```python
 from hirundo import (
     HirundoCSV,
@@ -152,7 +175,6 @@ results = test_dataset.check_run()
 print(results)
 ```
 
-
 Object detection example:
 
 ```python
@@ -187,21 +209,6 @@ test_dataset = OptimizationDataset(
             path="/BDD100K Val from Hirundo.zip/bdd100k/bdd100k.csv"
         ),
     ),
-    classes=[
-        "traffic light",
-        "traffic sign",
-        "car",
-        "pedestrian",
-        "bus",
-        "truck",
-        "rider",
-        "bicycle",
-        "motorcycle",
-        "train",
-        "other vehicle",
-        "other person",
-        "trailer",
-    ],
 )
 
 test_dataset.run_optimization()
@@ -209,8 +216,8 @@ results = test_dataset.check_run()
 print(results)
 ```
 
-Note: Currently we only support the main CPython release 3.9, 3.10 and 3.11. PyPy support may be introduced in the future.
+Note: Currently we only support the main CPython release 3.9, 3.10, 3.11, 3.12 & 3.13. PyPy support may be introduced in the future.
 
 ## Further documentation
 
-To learn more about how to use this library, please visit the [http://docs.hirundo.io/](documentation) or see the Google Colab examples.
+To learn more about how to use this library, please visit the [http://docs.hirundo.io/](documentation) or see the [Google Colab examples](https://github.com/Hirundo-io/hirundo-client/tree/main/notebooks).

{hirundo-0.1.16 → hirundo-0.1.18}/README.md

@@ -4,40 +4,62 @@ This package exposes access to Hirundo APIs for dataset optimization for Machine
 
 Dataset optimization is currently available for datasets labelled for classification and object detection.
 
-
 Support dataset storage configs include:
-   - Google Cloud (GCP) Storage
-   - Amazon Web Services (AWS) S3
-   - Git LFS (Large File Storage) repositories (e.g. GitHub or HuggingFace)
+
+- Google Cloud (GCP) Storage
+- Amazon Web Services (AWS) S3
+- Git LFS (Large File Storage) repositories (e.g. GitHub or HuggingFace)
+
+Note: This Python package must be used alongside a Hirundo server, either the SaaS platform, a custom VPC deployment or an on-premises installation.
 
 Optimizing a classification dataset
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Currently ``hirundo`` requires a CSV file with the following columns (all columns are required):
-   - ``image_path``: The location of the image within the dataset ``root``
-   - ``label``: The label of the image, i.e. which the class that was annotated for this image
+Currently `hirundo` requires a CSV file with the following columns (all columns are required):
+
+- `image_path`: The location of the image within the dataset `data_root_url`
+- `class_name`: The semantic label, i.e. the class name of the class that the image was annotated as belonging to
+
+And outputs two Pandas DataFrames with the dataset columns as well as:
+
+Suspect DataFrame (filename: `mislabel_suspects.csv`) columns:
 
-And outputs a CSV with the same columns and:
-   - ``suspect_level``: mislabel suspect level
-   - ``suggested_label``: suggested label
-   - ``suggested_label_conf``: suggested label confidence
+- ``suspect_score``: mislabel suspect score
+- ``suspect_level``: mislabel suspect level
+- ``suspect_rank``: mislabel suspect ranking
+- ``suggested_class_name``: suggested semantic label
+- ``suggested_class_conf``: suggested semantic label confidence
+
+Errors and warnings DataFrame (filename: `invalid_data.csv`) columns:
+
+   - ``status``: status message (one of ``NO_LABELS`` / ``MISSING_IMAGE`` / ``INVALID_IMAGE``)
 
 Optimizing an object detection (OD) dataset
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Currently ``hirundo`` requires a CSV file with the following columns (all columns are required):
-   - ``image_path``: The location of the image within the dataset ``root``
-   - ``bbox_id``: The index of the bounding box within the dataset. Used to indicate label suspects
-   - ``label``: The label of the image, i.e. which the class that was annotated for this image
-   - ``x1``, ``y1``, ``x2``, ``y2``: The bounding box coordinates of the object within the image
 
-And outputs a CSV with the same columns and:
-   - ``suspect_level``: object mislabel suspect level
-   - ``suggested_label``: suggested object label
-   - ``suggested_label_conf``: suggested object label confidence
+- ``image_path``: The location of the image within the dataset ``data_root_url``
+- ``object_id``: The ID of the bounding box within the dataset. Used to indicate object suspects
+- ``class_name``: Object semantic label, i.e. the class name of the object that was annotated
+- ``xmin``: leftmost horizontal pixel coordinate of the object's bounding box
+- ``ymin``: uppermost vertical pixel coordinate of the object's bounding box
+- ``xmax``: rightmost horizontal pixel coordinate of the object's bounding box
+- ``ymax``: lowermost vertical pixel coordinate of the object's bounding box
 
-Note: This Python package must be used alongside a Hirundo server, either the SaaS platform, a custom VPC deployment or an on-premises installation.
 
+And outputs two Pandas DataFrames with the dataset columns as well as:
+
+Suspect DataFrame (filename: `mislabel_suspects.csv`) columns:
+
+- ``suspect_score``: object mislabel suspect score
+- ``suspect_level``: object mislabel suspect level
+- ``suspect_rank``: object mislabel suspect ranking
+- ``suggested_class_name``: suggested object semantic label
+- ``suggested_class_conf``: suggested object semantic label confidence
+
+Errors and warnings DataFrame (filename: `invalid_data.csv`) columns:
+   - ``status``: status message (one of ``NO_LABELS`` / ``MISSING_IMAGE`` / ``INVALID_IMAGE`` / ``INVALID_BBOX`` / ``INVALID_BBOX_SIZE``)
 
 ## Installation
 
@@ -46,6 +68,7 @@ You can install the codebase with a simple `pip install hirundo` to install the
 ## Usage
 
 Classification example:
+
 ```python
 from hirundo import (
     HirundoCSV,
@@ -81,7 +104,6 @@ results = test_dataset.check_run()
 print(results)
 ```
 
-
 Object detection example:
 
 ```python
@@ -116,21 +138,6 @@ test_dataset = OptimizationDataset(
             path="/BDD100K Val from Hirundo.zip/bdd100k/bdd100k.csv"
         ),
     ),
-    classes=[
-        "traffic light",
-        "traffic sign",
-        "car",
-        "pedestrian",
-        "bus",
-        "truck",
-        "rider",
-        "bicycle",
-        "motorcycle",
-        "train",
-        "other vehicle",
-        "other person",
-        "trailer",
-    ],
 )
 
 test_dataset.run_optimization()
@@ -138,8 +145,8 @@ results = test_dataset.check_run()
 print(results)
 ```
 
-Note: Currently we only support the main CPython release 3.9, 3.10 and 3.11. PyPy support may be introduced in the future.
+Note: Currently we only support the main CPython release 3.9, 3.10, 3.11, 3.12 & 3.13. PyPy support may be introduced in the future.
 
 ## Further documentation
 
-To learn more about how to use this library, please visit the [http://docs.hirundo.io/](documentation) or see the Google Colab examples.
+To learn more about how to use this library, please visit the [http://docs.hirundo.io/](documentation) or see the [Google Colab examples](https://github.com/Hirundo-io/hirundo-client/tree/main/notebooks).

{hirundo-0.1.16 → hirundo-0.1.18}/hirundo/__init__.py

@@ -1,11 +1,9 @@
 from .dataset_enum import (
     DatasetMetadataType,
     LabelingType,
+    StorageTypes,
 )
 from .dataset_optimization import (
-    COCO,
-    YOLO,
-    HirundoCSV,
     HirundoError,
     OptimizationDataset,
     RunArgs,
@@ -13,26 +11,40 @@ from .dataset_optimization import (
 )
 from .dataset_optimization_results import DatasetOptimizationResults
 from .git import GitPlainAuth, GitRepo, GitSSHAuth
+from .labeling import (
+    COCO,
+    YOLO,
+    HirundoCSV,
+    KeylabsAuth,
+    KeylabsObjDetImages,
+    KeylabsObjDetVideo,
+    KeylabsObjSegImages,
+    KeylabsObjSegVideo,
+)
 from .storage import (
     StorageConfig,
     StorageGCP,
     # StorageAzure,  TODO: Azure storage is coming soon
     StorageGit,
     StorageS3,
-    StorageTypes,
 )
 from .unzip import load_df, load_from_zip
 
 __all__ = [
     "COCO",
     "YOLO",
-    "HirundoCSV",
     "HirundoError",
+    "HirundoCSV",
+    "KeylabsAuth",
+    "KeylabsObjDetImages",
+    "KeylabsObjDetVideo",
+    "KeylabsObjSegImages",
+    "KeylabsObjSegVideo",
     "OptimizationDataset",
     "RunArgs",
     "VisionRunArgs",
-    "LabelingType",
     "DatasetMetadataType",
+    "LabelingType",
     "GitPlainAuth",
     "GitRepo",
     "GitSSHAuth",
@@ -47,4 +59,4 @@ __all__ = [
     "load_from_zip",
 ]
 
-__version__ = "0.1.16"
+__version__ = "0.1.18"

hirundo-0.1.18/hirundo/_constraints.py

@@ -0,0 +1,164 @@
+import re
+import typing
+from typing import TYPE_CHECKING
+
+from hirundo._urls import (
+    LENGTH_CONSTRAINTS,
+    STORAGE_PATTERNS,
+)
+from hirundo.dataset_enum import DatasetMetadataType, LabelingType, StorageTypes
+from hirundo.labeling import COCO, YOLO, HirundoCSV, Keylabs
+
+if TYPE_CHECKING:
+    from hirundo._urls import HirundoUrl
+    from hirundo.dataset_optimization import LabelingInfo
+    from hirundo.storage import (
+        ResponseStorageConfig,
+        StorageConfig,
+        StorageGCP,
+        StorageGCPOut,
+        StorageS3,
+        StorageS3Out,
+    )
+
+LABELING_TYPES_TO_DATASET_METADATA_TYPES = {
+    LabelingType.SINGLE_LABEL_CLASSIFICATION: [
+        DatasetMetadataType.HIRUNDO_CSV,
+    ],
+    LabelingType.OBJECT_DETECTION: [
+        DatasetMetadataType.HIRUNDO_CSV,
+        DatasetMetadataType.COCO,
+        DatasetMetadataType.YOLO,
+        DatasetMetadataType.KeylabsObjDetImages,
+        DatasetMetadataType.KeylabsObjDetVideo,
+    ],
+    LabelingType.OBJECT_SEGMENTATION: [
+        DatasetMetadataType.HIRUNDO_CSV,
+        DatasetMetadataType.KeylabsObjSegImages,
+        DatasetMetadataType.KeylabsObjSegVideo,
+    ],
+    LabelingType.SEMANTIC_SEGMENTATION: [
+        DatasetMetadataType.HIRUNDO_CSV,
+    ],
+    LabelingType.PANOPTIC_SEGMENTATION: [
+        DatasetMetadataType.HIRUNDO_CSV,
+    ],
+    LabelingType.SPEECH_TO_TEXT: [
+        DatasetMetadataType.HIRUNDO_CSV,
+    ],
+}
+
+
+def validate_s3_url(str_url: str, s3_config: "StorageS3 | StorageS3Out"):
+    if (
+        len(str_url) < LENGTH_CONSTRAINTS[StorageTypes.S3]["min_length"]
+        or len(str_url) > LENGTH_CONSTRAINTS[StorageTypes.S3]["max_length"]
+    ):
+        raise ValueError("S3 URL must be between 8 and 1023 characters")
+    elif not re.match(STORAGE_PATTERNS[StorageTypes.S3], str_url):
+        raise ValueError(
+            f"Invalid S3 URL. Pattern must match: {STORAGE_PATTERNS[StorageTypes.S3]}"
+        )
+    elif not str_url.startswith(f"{s3_config.bucket_url}/"):
+        raise ValueError(f"S3 URL must start with {s3_config.bucket_url}/")
+
+
+def validate_gcp_url(str_url: str, gcp_config: "StorageGCP | StorageGCPOut"):
+    matches = re.match(STORAGE_PATTERNS[StorageTypes.GCP], str_url)
+    if (
+        len(str_url) < LENGTH_CONSTRAINTS[StorageTypes.GCP]["min_length"]
+        or len(str_url) > LENGTH_CONSTRAINTS[StorageTypes.GCP]["max_length"]
+    ):
+        raise ValueError(
+            f"GCP URL must be between {LENGTH_CONSTRAINTS[StorageTypes.GCP]['min_length']}"
+            + f" and {LENGTH_CONSTRAINTS[StorageTypes.GCP]['max_length']} characters"
+        )
+    elif not matches:
+        raise ValueError(
+            f"Invalid GCP URL. Pattern must match: {STORAGE_PATTERNS[StorageTypes.GCP]}"
+        )
+    elif (
+        matches
+        and len(matches.group(1))
+        > LENGTH_CONSTRAINTS[StorageTypes.GCP]["bucket_max_length"]
+    ):
+        raise ValueError(
+            f"GCP bucket name must be between {LENGTH_CONSTRAINTS[StorageTypes.GCP]['bucket_min_length']} "
+            + f"and {LENGTH_CONSTRAINTS[StorageTypes.GCP]['bucket_max_length']} characters"
+        )
+    elif not str_url.startswith(f"gs://{gcp_config.bucket_name}/"):
+        raise ValueError(f"GCP URL must start with gs://{gcp_config.bucket_name}")
+
+
+def validate_url(
+    url: "HirundoUrl",
+    storage_config: "StorageConfig | ResponseStorageConfig",
+) -> "HirundoUrl":
+    s3_config = storage_config.s3
+    gcp_config = storage_config.gcp
+    git_config = storage_config.git
+    str_url = str(url)
+
+    if s3_config is not None:
+        validate_s3_url(str_url, s3_config)
+    elif gcp_config is not None:
+        validate_gcp_url(str_url, gcp_config)
+    elif (
+        git_config is not None
+        and not str_url.startswith("https://")
+        and not str_url.startswith("ssh://")
+    ):
+        raise ValueError("Git URL must start with https:// or ssh://")
+    elif storage_config.type == StorageTypes.LOCAL and not str_url.startswith(
+        "file:///datasets/"
+    ):
+        raise ValueError("Local URL must start with file:///datasets/")
+    return url
+
+
+def validate_labeling_type(
+    labeling_type: "LabelingType", labeling_info: "LabelingInfo"
+) -> None:
+    """
+    Validate that the labeling type is compatible with the labeling info
+
+    Args:
+        labeling_type: The type of labeling that will be performed
+        labeling_info: The labeling info to validate
+    """
+    dataset_metadata_types = LABELING_TYPES_TO_DATASET_METADATA_TYPES[labeling_type]
+    if labeling_info.type not in dataset_metadata_types:
+        raise ValueError(
+            f"Cannot use {labeling_info.type.name} labeling info with {labeling_type.name} datasets"
+        )
+
+
+def validate_labeling_info(
+    labeling_type: "LabelingType",
+    labeling_info: "typing.Union[LabelingInfo, list[LabelingInfo]]",
+    storage_config: "typing.Union[StorageConfig, ResponseStorageConfig]",
+) -> None:
+    """
+    Validate the labeling info for a dataset
+
+    Args:
+        labeling_type: The type of labeling that will be performed
+        labeling_info: The labeling info to validate
+        storage_config: The storage configuration for the dataset.
+            StorageConfig is used to validate the URLs in the labeling info
+    """
+    if isinstance(labeling_info, list):
+        for labeling in labeling_info:
+            validate_labeling_info(labeling_type, labeling, storage_config)
+        return
+    elif isinstance(labeling_info, HirundoCSV):
+        validate_url(labeling_info.csv_url, storage_config)
+    elif isinstance(labeling_info, COCO):
+        validate_url(labeling_info.json_url, storage_config)
+    elif isinstance(labeling_info, YOLO):
+        validate_url(labeling_info.labels_dir_url, storage_config)
+        if labeling_info.data_yaml_url is not None:
+            validate_url(labeling_info.data_yaml_url, storage_config)
+    elif isinstance(labeling_info, Keylabs):
+        validate_url(labeling_info.labels_dir_url, storage_config)
+    validate_labeling_type(labeling_type, labeling_info)

hirundo-0.1.16/hirundo/_constraints.py → hirundo-0.1.18/hirundo/_urls.py

@@ -3,6 +3,8 @@ from typing import Annotated
 from pydantic import StringConstraints, UrlConstraints
 from pydantic_core import Url
 
+from hirundo.dataset_enum import StorageTypes
+
 S3BucketUrl = Annotated[
     str,
     StringConstraints(
@@ -21,12 +23,16 @@ StorageConfigName = Annotated[
     ),
 ]
 
-S3_MIN_LENGTH = 8
-S3_MAX_LENGTH = 1023
-S3_PATTERN = r"s3://[a-zA-Z0-9.-]{3,64}/[a-zA-Z0-9.-/]+"
-GCP_MIN_LENGTH = 8
-GCP_MAX_LENGTH = 1023
-GCP_PATTERN = r"gs://[a-zA-Z0-9.-]{3,64}/[a-zA-Z0-9.-/]+"
+STORAGE_PATTERNS: dict[StorageTypes, str] = {
+    StorageTypes.S3: r"^s3:\/\/[a-z0-9\.\-]{3,63}/[a-zA-Z0-9!\-\/_\.\*'\(\)]+$",
+    StorageTypes.GCP: r"^gs:\/\/([a-z0-9][a-z0-9_-]{1,61}[a-z0-9](\.[a-z0-9][a-z0-9_-]{1,61}[a-z0-9])*)\/[^\x00-\x1F\x7F-\x9F\r\n]*$",
+}
+
+
+LENGTH_CONSTRAINTS: dict[StorageTypes, dict] = {
+    StorageTypes.S3: {"min_length": 8, "max_length": 1023, "bucket_max_length": None},
+    StorageTypes.GCP: {"min_length": 8, "max_length": 1023, "bucket_max_length": 222},
+}
 
 RepoUrl = Annotated[
     Url,

hirundo-0.1.18/hirundo/dataset_enum.py

@@ -0,0 +1,46 @@
+from enum import Enum
+
+
+class LabelingType(str, Enum):
+    """
+    Enum indicate what type of labeling is used for the given dataset.
+    Supported types are:
+    """
+
+    SINGLE_LABEL_CLASSIFICATION = "SingleLabelClassification"
+    OBJECT_DETECTION = "ObjectDetection"
+    SPEECH_TO_TEXT = "SpeechToText"
+    OBJECT_SEGMENTATION = "ObjectSegmentation"
+    SEMANTIC_SEGMENTATION = "SemanticSegmentation"
+    PANOPTIC_SEGMENTATION = "PanopticSegmentation"
+
+
+class DatasetMetadataType(str, Enum):
+    """
+    Enum indicate what type of metadata is provided for the given dataset.
+    Supported types are:
+    """
+
+    HIRUNDO_CSV = "HirundoCSV"
+    COCO = "COCO"
+    YOLO = "YOLO"
+    KeylabsObjDetImages = "KeylabsObjDetImages"
+    KeylabsObjDetVideo = "KeylabsObjDetVideo"
+    KeylabsObjSegImages = "KeylabsObjSegImages"
+    KeylabsObjSegVideo = "KeylabsObjSegVideo"
+
+
+class StorageTypes(str, Enum):
+    """
+    Enum for the different types of storage configs.
+    Supported types are:
+    """
+
+    S3 = "S3"
+    GCP = "GCP"
+    # AZURE = "Azure"  TODO: Azure storage config is coming soon
+    GIT = "Git"
+    LOCAL = "Local"
+    """
+    Local storage config is only supported for on-premises installations.
+    """