datacosmos 0.0.1__tar.gz → 0.0.3__tar.gz

{datacosmos-0.0.1 → datacosmos-0.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.2
+Metadata-Version: 2.4
 Name: datacosmos
-Version: 0.0.1
+Version: 0.0.3
 Summary: A library for interacting with DataCosmos from Python code
 Author-email: Open Cosmos <support@open-cosmos.com>
 Classifier: Programming Language :: Python :: 3
@@ -20,3 +20,4 @@ Requires-Dist: pytest==7.2.0; extra == "dev"
 Requires-Dist: bandit[toml]==1.7.4; extra == "dev"
 Requires-Dist: isort==5.11.4; extra == "dev"
 Requires-Dist: pydocstyle==6.1.1; extra == "dev"
+Dynamic: license-file

datacosmos-0.0.3/README.md

@@ -0,0 +1,209 @@
+# DataCosmos SDK
+
+## Overview
+
+The **DataCosmos SDK** enables Open Cosmos customers to interact with the **DataCosmos APIs** for efficient data management and retrieval. It provides authentication handling, HTTP request utilities, and a client for interacting with the **STAC API** (SpatioTemporal Asset Catalog).
+
+## Installation
+
+Install the SDK using **pip**:
+
+```sh
+pip install datacosmos=={version}
+```
+
+## Initializing the Client
+
+To start using the SDK, initialize the client. The easiest way to do this is by loading the configuration from a YAML file. Alternatively, you can manually instantiate the Config object or use environment variables.
+
+### Default Initialization (Recommended)
+
+By default, the client loads configuration from a YAML file (`config/config.yaml`).
+
+```python
+from datacosmos.datacosmos_client import DatacosmosClient
+
+client = DatacosmosClient()
+```
+
+### Loading from YAML (Recommended)
+
+Create a YAML file (`config/config.yaml`) with the following content:
+
+```yaml
+authentication:
+    client_id: {client_id}  
+    client_secret: {client_secret}
+```
+
+The client will automatically read this file when initialized.
+
+### Loading from Environment Variables
+
+Set the following environment variables:
+
+```sh
+export OC_AUTH_CLIENT_ID={client_id}
+export OC_AUTH_CLIENT_SECRET={client_secret}
+```
+
+The client will automatically read these values when initialized.
+
+### Manual Instantiation
+
+If manually instantiating `Config`, default values are now applied where possible.
+
+```python
+from config.config import Config
+from config.models.m2m_authentication_config import M2MAuthenticationConfig
+from config.models.url import URL
+
+config = Config(
+    authentication=M2MAuthenticationConfig(
+        client_id="your-client-id",
+        client_secret="your-client-secret"
+    )
+)
+
+client = DatacosmosClient(config=config)
+```
+
+### Configuration Options and Defaults
+
+| Setting                      | Default Value                                     | Override Method |
+|------------------------------|-------------------------------------------------|----------------|
+| `authentication.type`        | `m2m`                                           | YAML / ENV     |
+| `authentication.client_id`   | _Required in manual instantiation_              | YAML / ENV     |
+| `authentication.client_secret` | _Required in manual instantiation_            | YAML / ENV     |
+| `stac.protocol`              | `https`                                         | YAML / ENV     |
+| `stac.host`                  | `app.open-cosmos.com`                           | YAML / ENV     |
+| `stac.port`                  | `443`                                           | YAML / ENV     |
+| `stac.path`                  | `/api/data/v0/stac`                             | YAML / ENV     |
+| `datacosmos_cloud_storage.protocol` | `https`                                 | YAML / ENV     |
+| `datacosmos_cloud_storage.host`     | `app.open-cosmos.com`                   | YAML / ENV     |
+| `datacosmos_cloud_storage.port`     | `443`                                   | YAML / ENV     |
+| `datacosmos_cloud_storage.path`     | `/api/data/v0/storage`                 | YAML / ENV     |
+| `mission_id`                | `0`                                             | YAML / ENV     |
+| `environment`               | `test`                                          | YAML / ENV     |
+
+## STAC Client
+
+The `STACClient` enables interaction with the STAC API, allowing for searching, retrieving, creating, updating, and deleting STAC items and collections.
+
+### Initialize STACClient
+
+```python
+from datacosmos.datacosmos_client import DatacosmosClient
+from datacosmos.stac.stac_client import STACClient
+
+client = DatacosmosClient()
+stac_client = STACClient(client)
+```
+
+### STACClient Methods
+
+#### 1. Fetch a Collection
+
+```python
+collection = stac_client.fetch_collection("test-collection")
+```
+
+#### 2. Fetch All Collections
+
+```python
+collections = list(stac_client.fetch_all_collections())
+```
+
+#### 3. Create a Collection
+
+```python
+from pystac import Collection
+
+new_collection = Collection(
+    id="test-collection",
+    title="Test Collection",
+    description="This is a test collection",
+    license="proprietary",
+    extent={
+        "spatial": {"bbox": [[-180, -90, 180, 90]]},
+        "temporal": {"interval": [["2023-01-01T00:00:00Z", None]]},
+    },
+)
+
+stac_client.create_collection(new_collection)
+```
+
+#### 4. Update a Collection
+
+```python
+from datacosmos.stac.collection.models.collection_update import CollectionUpdate
+
+update_data = CollectionUpdate(
+    title="Updated Collection Title",
+    description="Updated description",
+)
+
+stac_client.update_collection("test-collection", update_data)
+```
+
+#### 5. Delete a Collection
+
+```python
+stac_client.delete_collection("test-collection")
+```
+
+### Uploading Files and Registering STAC Items
+
+You can use the `DatacosmosUploader` class to upload files to the DataCosmos cloud storage and register a STAC item.
+
+#### Upload Files and Register STAC Item
+
+```python
+from datacosmos.uploader.datacosmos_uploader import DatacosmosUploader
+
+uploader = DatacosmosUploader(client)
+item_json_file_path = "/path/to/stac_item.json"
+uploader.upload_and_register_item(item_json_file_path)
+```
+
+## Error Handling
+
+Use `try-except` blocks to handle API errors gracefully:
+
+```python
+try:
+    data = client.get_data("dataset_id")
+    print(data)
+except Exception as e:
+    print(f"An error occurred: {e}")
+```
+
+## Contributing
+
+To contribute:
+
+1. Fork the repository.
+2. Create a feature branch.
+3. Submit a pull request.
+
+### Development Setup
+
+Use `uv` for dependency management:
+
+```sh
+pip install uv
+uv venv
+uv pip install -r pyproject.toml .[dev]
+source .venv/bin/activate
+```
+
+Before making changes, run:
+
+```sh
+black .
+isort .
+ruff check .
+pydocstyle .
+bandit -r -c pyproject.toml .
+pytest
+```

datacosmos-0.0.3/config/__init__.py

@@ -0,0 +1,5 @@
+"""Configuration package for the Datacosmos SDK.
+
+This package includes modules for loading and managing authentication
+configurations.
+"""

datacosmos-0.0.3/config/config.py

@@ -0,0 +1,195 @@
+"""Configuration module for the Datacosmos SDK.
+
+Handles configuration management using Pydantic and Pydantic Settings.
+It loads default values, allows overrides via YAML configuration files,
+and supports environment variable-based overrides.
+"""
+
+import os
+from typing import ClassVar, Literal, Optional
+
+import yaml
+from pydantic import field_validator
+from pydantic_settings import BaseSettings, SettingsConfigDict
+
+from config.models.m2m_authentication_config import M2MAuthenticationConfig
+from config.models.url import URL
+
+
+class Config(BaseSettings):
+    """Centralized configuration for the Datacosmos SDK."""
+
+    model_config = SettingsConfigDict(
+        env_nested_delimiter="__",
+        nested_model_default_partial_update=True,
+        extra="allow",
+    )
+
+    authentication: Optional[M2MAuthenticationConfig] = None
+    stac: Optional[URL] = None
+    datacosmos_cloud_storage: Optional[URL] = None
+    mission_id: int = 0
+    environment: Literal["local", "test", "prod"] = "test"
+
+    DEFAULT_AUTH_TYPE: ClassVar[str] = "m2m"
+    DEFAULT_AUTH_TOKEN_URL: ClassVar[str] = "https://login.open-cosmos.com/oauth/token"
+    DEFAULT_AUTH_AUDIENCE: ClassVar[str] = "https://beeapp.open-cosmos.com"
+
+    @classmethod
+    def from_yaml(cls, file_path: str = "config/config.yaml") -> "Config":
+        """Load configuration from a YAML file and override defaults.
+
+        Args:
+            file_path (str): The path to the YAML configuration file.
+
+        Returns:
+            Config: An instance of the Config class with loaded settings.
+        """
+        config_data: dict = {}
+        if os.path.exists(file_path):
+            with open(file_path, "r") as f:
+                yaml_data = yaml.safe_load(f) or {}
+                # Remove empty values from YAML to avoid overwriting with `None`
+                config_data = {
+                    key: value
+                    for key, value in yaml_data.items()
+                    if value not in [None, ""]
+                }
+
+        return cls(**config_data)
+
+    @classmethod
+    def from_env(cls) -> "Config":
+        """Load configuration from environment variables.
+
+        Returns:
+            Config: An instance of the Config class with settings loaded from environment variables.
+        """
+        authentication_config = M2MAuthenticationConfig(
+            type=os.getenv("OC_AUTH_TYPE", cls.DEFAULT_AUTH_TYPE),
+            client_id=os.getenv("OC_AUTH_CLIENT_ID"),
+            client_secret=os.getenv("OC_AUTH_CLIENT_SECRET"),
+            token_url=os.getenv("OC_AUTH_TOKEN_URL", cls.DEFAULT_AUTH_TOKEN_URL),
+            audience=os.getenv("OC_AUTH_AUDIENCE", cls.DEFAULT_AUTH_AUDIENCE),
+        )
+
+        stac_config = URL(
+            protocol=os.getenv("OC_STAC_PROTOCOL", "https"),
+            host=os.getenv("OC_STAC_HOST", "app.open-cosmos.com"),
+            port=int(os.getenv("OC_STAC_PORT", "443")),
+            path=os.getenv("OC_STAC_PATH", "/api/data/v0/stac"),
+        )
+
+        datacosmos_cloud_storage_config = URL(
+            protocol=os.getenv("DC_CLOUD_STORAGE_PROTOCOL", "https"),
+            host=os.getenv("DC_CLOUD_STORAGE_HOST", "app.open-cosmos.com"),
+            port=int(os.getenv("DC_CLOUD_STORAGE_PORT", "443")),
+            path=os.getenv("DC_CLOUD_STORAGE_PATH", "/api/data/v0/storage"),
+        )
+
+        return cls(
+            authentication=authentication_config,
+            stac=stac_config,
+            datacosmos_cloud_storage=datacosmos_cloud_storage_config,
+            mission_id=int(os.getenv("MISSION_ID", "0")),
+            environment=os.getenv("ENVIRONMENT", "test"),
+        )
+
+    @field_validator("authentication", mode="before")
+    @classmethod
+    def validate_authentication(
+        cls, auth_data: Optional[dict]
+    ) -> M2MAuthenticationConfig:
+        """Ensure authentication is provided and apply defaults.
+
+        Args:
+            auth_data (Optional[dict]): The authentication config as a dictionary.
+
+        Returns:
+            M2MAuthenticationConfig: The validated authentication configuration.
+
+        Raises:
+            ValueError: If authentication is missing or required fields are not set.
+        """
+        if not auth_data:
+            return cls.apply_auth_defaults(M2MAuthenticationConfig())
+
+        auth = cls.parse_auth_config(auth_data)
+        auth = cls.apply_auth_defaults(auth)
+
+        cls.check_required_auth_fields(auth)
+        return auth
+
+    @staticmethod
+    def apply_auth_defaults(auth: M2MAuthenticationConfig) -> M2MAuthenticationConfig:
+        """Apply default authentication values if they are missing."""
+        auth.type = auth.type or Config.DEFAULT_AUTH_TYPE
+        auth.token_url = auth.token_url or Config.DEFAULT_AUTH_TOKEN_URL
+        auth.audience = auth.audience or Config.DEFAULT_AUTH_AUDIENCE
+        return auth
+
+    @classmethod
+    def parse_auth_config(cls, auth_data: dict) -> M2MAuthenticationConfig:
+        """Parse authentication config from a dictionary."""
+        return M2MAuthenticationConfig(
+            type=auth_data.get("type", cls.DEFAULT_AUTH_TYPE),
+            token_url=auth_data.get("token_url", cls.DEFAULT_AUTH_TOKEN_URL),
+            audience=auth_data.get("audience", cls.DEFAULT_AUTH_AUDIENCE),
+            client_id=auth_data.get("client_id"),
+            client_secret=auth_data.get("client_secret"),
+        )
+
+    @staticmethod
+    def check_required_auth_fields(auth: M2MAuthenticationConfig):
+        """Ensure required fields (client_id, client_secret) are provided."""
+        missing_fields = [
+            field
+            for field in ("client_id", "client_secret")
+            if not getattr(auth, field)
+        ]
+        if missing_fields:
+            raise ValueError(
+                f"Missing required authentication fields: {', '.join(missing_fields)}"
+            )
+
+    @field_validator("stac", mode="before")
+    @classmethod
+    def validate_stac(cls, stac_config: Optional[URL]) -> URL:
+        """Ensure STAC configuration has a default if not explicitly set.
+
+        Args:
+            stac_config (Optional[URL]): The STAC config to validate.
+
+        Returns:
+            URL: The validated STAC configuration.
+        """
+        if stac_config is None:
+            return URL(
+                protocol="https",
+                host="app.open-cosmos.com",
+                port=443,
+                path="/api/data/v0/stac",
+            )
+        return stac_config
+
+    @field_validator("datacosmos_cloud_storage", mode="before")
+    @classmethod
+    def validate_datacosmos_cloud_storage(
+        cls, datacosmos_cloud_storage_config: Optional[URL]
+    ) -> URL:
+        """Ensure datacosmos cloud storage configuration has a default if not explicitly set.
+
+        Args:
+            datacosmos_cloud_storage_config (Optional[URL]): The datacosmos cloud storage config to validate.
+
+        Returns:
+            URL: The validated datacosmos cloud storage configuration.
+        """
+        if datacosmos_cloud_storage_config is None:
+            return URL(
+                protocol="https",
+                host="app.open-cosmos.com",
+                port=443,
+                path="/api/data/v0/storage",
+            )
+        return datacosmos_cloud_storage_config

datacosmos-0.0.3/config/models/__init__.py

@@ -0,0 +1 @@
+"""Models for configuration settings."""

datacosmos-0.0.3/config/models/m2m_authentication_config.py

@@ -0,0 +1,23 @@
+"""Module for configuring machine-to-machine (M2M) authentication.
+
+Used when running scripts in the cluster that require automated authentication
+without user interaction.
+"""
+
+from typing import Literal
+
+from pydantic import BaseModel
+
+
+class M2MAuthenticationConfig(BaseModel):
+    """Configuration for machine-to-machine authentication.
+
+    This is used when running scripts in the cluster that require authentication
+    with client credentials.
+    """
+
+    type: Literal["m2m"]
+    client_id: str
+    token_url: str
+    audience: str
+    client_secret: str

datacosmos-0.0.3/config/models/url.py

@@ -0,0 +1,35 @@
+"""Module defining a structured URL configuration model.
+
+Ensures that URLs contain required components such as protocol, host,
+port, and path.
+"""
+
+from pydantic import BaseModel
+
+from datacosmos.utils.url import URL as DomainURL
+
+
+class URL(BaseModel):
+    """Generic configuration model for a URL.
+
+    This class provides attributes to store URL components and a method
+    to convert them into a `DomainURL` instance.
+    """
+
+    protocol: str
+    host: str
+    port: int
+    path: str
+
+    def as_domain_url(self) -> DomainURL:
+        """Convert the URL instance to a `DomainURL` object.
+
+        Returns:
+            DomainURL: A domain-specific URL object.
+        """
+        return DomainURL(
+            protocol=self.protocol,
+            host=self.host,
+            port=self.port,
+            base=self.path,
+        )

datacosmos-0.0.3/datacosmos/exceptions/__init__.py

@@ -0,0 +1 @@
+"""Exceptions for the datacosmos package."""

datacosmos-0.0.3/datacosmos/exceptions/datacosmos_exception.py

@@ -0,0 +1,27 @@
+"""Base exception class for all Datacosmos SDK exceptions."""
+
+from typing import Optional
+
+from requests import Response
+from requests.exceptions import RequestException
+
+
+class DatacosmosException(RequestException):
+    """Base exception class for all Datacosmos SDK exceptions."""
+
+    def __init__(self, message: str, response: Optional[Response] = None):
+        """Initialize DatacosmosException.
+
+        Args:
+            message (str): The error message.
+            response (Optional[Response]): The HTTP response object, if available.
+        """
+        self.response = response
+        self.status_code = response.status_code if response else None
+        self.details = response.text if response else None
+        full_message = (
+            f"{message} (Status: {self.status_code}, Details: {self.details})"
+            if response
+            else message
+        )
+        super().__init__(full_message)

datacosmos-0.0.3/datacosmos/stac/__init__.py

@@ -0,0 +1,5 @@
+"""STAC package for interacting with the STAC API, providing query and fetch functionalities.
+
+It enables interaction with STAC (SpatioTemporal Asset Catalog) services
+using an authenticated Datacosmos client.
+"""

datacosmos-0.0.3/datacosmos/stac/collection/__init__.py

@@ -0,0 +1,4 @@
+"""STAC package for interacting with collections from the STAC API, providing query and fetch functionalities.
+
+It enables interaction with collections from the STAC using an authenticated Datacosmos client.
+"""

datacosmos-0.0.3/datacosmos/stac/collection/collection_client.py

@@ -0,0 +1,149 @@
+"""Handles operations related to STAC collections."""
+
+from typing import Generator, Optional
+
+from pystac import Collection, Extent, SpatialExtent, TemporalExtent
+from pystac.utils import str_to_datetime
+
+from datacosmos.datacosmos_client import DatacosmosClient
+from datacosmos.stac.collection.models.collection_update import CollectionUpdate
+from datacosmos.utils.http_response.check_api_response import check_api_response
+
+
+class CollectionClient:
+    """Handles operations related to STAC collections."""
+
+    def __init__(self, client: DatacosmosClient):
+        """Initialize the CollectionClient with a DatacosmosClient."""
+        self.client = client
+        self.base_url = client.config.stac.as_domain_url()
+
+    def fetch_collection(self, collection_id: str) -> Collection:
+        """Fetch details of an existing STAC collection."""
+        url = self.base_url.with_suffix(f"/collections/{collection_id}")
+        response = self.client.get(url)
+        check_api_response(response)
+        return Collection.from_dict(response.json())
+
+    def create_collection(self, collection: Collection) -> None:
+        """Create a new STAC collection.
+
+        Args:
+            collection (Collection): The STAC collection to create.
+
+        Raises:
+            InvalidRequest: If the collection data is malformed.
+        """
+        if isinstance(collection.extent, dict):
+            spatial_data = collection.extent.get("spatial", {}).get("bbox", [[]])
+            temporal_data = collection.extent.get("temporal", {}).get("interval", [[]])
+
+            # Convert string timestamps to datetime objects
+            parsed_temporal = []
+            for interval in temporal_data:
+                start = str_to_datetime(interval[0]) if interval[0] else None
+                end = (
+                    str_to_datetime(interval[1])
+                    if len(interval) > 1 and interval[1]
+                    else None
+                )
+                parsed_temporal.append([start, end])
+
+            collection.extent = Extent(
+                spatial=SpatialExtent(spatial_data),
+                temporal=TemporalExtent(parsed_temporal),
+            )
+
+        url = self.base_url.with_suffix("/collections")
+        response = self.client.post(url, json=collection.to_dict())
+        check_api_response(response)
+
+    def update_collection(
+        self, collection_id: str, update_data: CollectionUpdate
+    ) -> None:
+        """Update an existing STAC collection."""
+        url = self.base_url.with_suffix(f"/collections/{collection_id}")
+        response = self.client.patch(
+            url, json=update_data.model_dump(by_alias=True, exclude_none=True)
+        )
+        check_api_response(response)
+
+    def delete_collection(self, collection_id: str) -> None:
+        """Delete a STAC collection by its ID."""
+        url = self.base_url.with_suffix(f"/collections/{collection_id}")
+        response = self.client.delete(url)
+        check_api_response(response)
+
+    def fetch_all_collections(self) -> Generator[Collection, None, None]:
+        """Fetch all STAC collections with pagination support."""
+        url = self.base_url.with_suffix("/collections")
+        params = {"limit": 10}
+
+        while True:
+            data = self._fetch_collections_page(url, params)
+            yield from self._parse_collections(data)
+
+            next_cursor = self._get_next_pagination_cursor(data)
+            if not next_cursor:
+                break
+
+            params["cursor"] = next_cursor
+
+    def _fetch_collections_page(self, url: str, params: dict) -> dict:
+        """Fetch a single page of collections from the API."""
+        response = self.client.get(url, params=params)
+        check_api_response(response)
+
+        data = response.json()
+
+        if isinstance(data, list):
+            return {"collections": data}
+
+        return data
+
+    def _parse_collections(self, data: dict) -> Generator[Collection, None, None]:
+        """Convert API response data to STAC Collection objects, ensuring required fields exist."""
+        return (
+            Collection.from_dict(
+                {
+                    **collection,
+                    "type": collection.get("type", "Collection"),
+                    "id": collection.get("id", ""),
+                    "stac_version": collection.get("stac_version", "1.0.0"),
+                    "extent": collection.get(
+                        "extent",
+                        {"spatial": {"bbox": []}, "temporal": {"interval": []}},
+                    ),
+                    "links": collection.get("links", []) or [],
+                    "properties": collection.get("properties", {}),
+                }
+            )
+            for collection in data.get("collections", [])
+            if collection.get("type") == "Collection"
+        )
+
+    def _get_next_pagination_cursor(self, data: dict) -> Optional[str]:
+        """Extract the next pagination token from the response."""
+        next_href = self._get_next_link(data)
+        return self._extract_pagination_token(next_href) if next_href else None
+
+    def _get_next_link(self, data: dict) -> Optional[str]:
+        """Extract the next page link from the response."""
+        next_link = next(
+            (link for link in data.get("links", []) if link.get("rel") == "next"), None
+        )
+        return next_link.get("href", "") if next_link else None
+
+    def _extract_pagination_token(self, next_href: str) -> Optional[str]:
+        """Extract the pagination token from the next link URL.
+
+        Args:
+            next_href (str): The next page URL.
+
+        Returns:
+            Optional[str]: The extracted token, or None if parsing fails.
+        """
+        try:
+            return next_href.split("?")[1].split("=")[-1]
+        except (IndexError, AttributeError):
+            raise InvalidRequest(f"Failed to parse pagination token from {next_href}")

datacosmos-0.0.3/datacosmos/stac/collection/models/__init__.py

@@ -0,0 +1 @@
+"""Models for the Collection Client."""