ndp-ep 0.1.0__tar.gz

ndp_ep-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 NDP EP Team
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ndp_ep-0.1.0/PKG-INFO

@@ -0,0 +1,217 @@
+Metadata-Version: 2.4
+Name: ndp-ep
+Version: 0.1.0
+Summary: Python client library for NDP EP API
+Author-email: Your Name <your.email@example.com>
+License: MIT
+Project-URL: Homepage, https://github.com/sci-ndp/ndp-ep-py
+Project-URL: Repository, https://github.com/sci-ndp/ndp-ep-py
+Project-URL: Documentation, https://github.com/sci-ndp/ndp-ep-py#readme
+Project-URL: Bug Tracker, https://github.com/sci-ndp/ndp-ep-py/issues
+Keywords: api,client,ndp,ep
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.25.0
+Requires-Dist: urllib3>=1.26.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+Requires-Dist: pytest-mock>=3.10.0; extra == "dev"
+Requires-Dist: requests-mock>=1.9.0; extra == "dev"
+Requires-Dist: black>=22.0.0; extra == "dev"
+Requires-Dist: flake8>=5.0.0; extra == "dev"
+Requires-Dist: mypy>=1.0.0; extra == "dev"
+Requires-Dist: twine>=4.0.0; extra == "dev"
+Requires-Dist: build>=0.10.0; extra == "dev"
+Dynamic: license-file
+
+# ndp-ep
+
+[![CI](https://github.com/sci-ndp/ndp-ep-py/workflows/CI/badge.svg)](https://github.com/sci-ndp/ndp-ep-py/actions)
+[![codecov](https://codecov.io/gh/sci-ndp/ndp-ep-py/branch/main/graph/badge.svg)](https://codecov.io/gh/sci-ndp/ndp-ep-py)
+[![PyPI version](https://badge.fury.io/py/ndp-ep.svg)](https://badge.fury.io/py/ndp-ep)
+[![Python versions](https://img.shields.io/pypi/pyversions/ndp-ep.svg)](https://pypi.org/project/ndp-ep/)
+
+A Python client library for interacting with the NDP EP API. This library provides a simple and intuitive interface for managing datasets, organizations, resources, and services through the API.
+
+## Features
+
+- **Complete API Coverage**: Support for all API endpoints including Kafka topics, S3 resources, URL resources, organizations, and services
+- **Authentication**: Token-based and username/password authentication
+- **Search Functionality**: Advanced search capabilities across datasets and resources
+- **Error Handling**: Comprehensive error handling with meaningful error messages
+- **Type Hints**: Full type hint support for better IDE integration
+- **Testing**: Extensive test coverage (>70%) with unit and integration tests
+
+## Installation
+
+```bash
+pip install ndp-ep
+```
+
+## Quick Start
+
+```python
+from ndp_ep import APIClient
+
+# Initialize client with token
+client = APIClient(
+    base_url="https://your-api-endpoint.com",
+    token="your-access-token"
+)
+
+# Or with username/password
+client = APIClient(
+    base_url="https://your-api-endpoint.com",
+    username="your-username",
+    password="your-password"
+)
+
+# List organizations
+organizations = client.list_organizations()
+print(organizations)
+
+# Search datasets
+results = client.search_datasets(
+    terms=["climate", "temperature"],
+    server="global"
+)
+
+# Register a new organization
+org_data = {
+    "name": "my_organization",
+    "title": "My Organization",
+    "description": "A sample organization"
+}
+response = client.register_organization(org_data)
+
+# Register a service
+service_data = {
+    "service_name": "user_auth_api",
+    "service_title": "User Authentication API",
+    "owner_org": "services",
+    "service_url": "https://api.example.com/auth",
+    "service_type": "API",
+    "notes": "RESTful API for user authentication"
+}
+response = client.register_service(service_data)
+```
+
+## API Coverage
+
+### Authentication
+- Token-based authentication
+- Username/password authentication
+
+### Organizations
+- `list_organizations()` - List all organizations
+- `register_organization()` - Create new organization
+- `delete_organization()` - Delete organization
+
+### Datasets and Resources
+- `search_datasets()` - Search datasets with advanced filters
+- `advanced_search()` - Advanced search with POST method
+- `register_url()` - Register URL resources
+- `register_s3_link()` - Register S3 resources
+- `register_kafka_topic()` - Register Kafka topics
+- `register_general_dataset()` - Register general datasets
+- `update_url_resource()` - Update URL resources
+- `update_s3_resource()` - Update S3 resources
+- `update_kafka_topic()` - Update Kafka topics
+- `update_general_dataset()` - Update general datasets (PUT)
+- `patch_general_dataset()` - Partially update general datasets (PATCH)
+- `delete_resource_by_id()` - Delete resource by ID
+- `delete_resource_by_name()` - Delete resource by name
+
+### Services
+- `register_service()` - Register new services
+
+### System Information
+- `get_kafka_details()` - Get Kafka connection details
+- `get_system_status()` - Check system status
+- `get_system_metrics()` - Get system metrics
+- `get_jupyter_details()` - Get Jupyter connection details
+
+## Development
+
+### Setting up development environment
+
+```bash
+# Clone the repository
+git clone https://github.com/sci-ndp/ndp-ep-py.git
+cd ndp-ep-py
+
+# Create virtual environment
+python -m venv venv
+source venv/bin/activate  # On Windows: venv\Scripts\activate
+
+# Install development dependencies
+pip install -e .
+pip install -r requirements-dev.txt
+```
+
+### Running tests
+
+```bash
+# Run all tests
+pytest
+
+# Run with coverage
+pytest --cov=ndp_ep --cov-report=html
+
+# Run specific test categories
+pytest -m unit          # Unit tests only
+pytest -m integration   # Integration tests only
+```
+
+### Code formatting and linting
+
+```bash
+# Format code
+black ndp_ep tests
+
+# Lint code
+flake8 ndp_ep
+
+# Type checking
+mypy ndp_ep
+```
+
+## Contributing
+
+1. Fork the repository
+2. Create a feature branch (`git checkout -b feature/amazing-feature`)
+3. Make your changes
+4. Add tests for your changes
+5. Ensure all tests pass and coverage is maintained
+6. Commit your changes (`git commit -m 'Add amazing feature'`)
+7. Push to the branch (`git push origin feature/amazing-feature`)
+8. Open a Pull Request
+
+## License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+## Changelog
+
+### v0.1.0
+- Initial release
+- Complete API coverage for all endpoints
+- Authentication support (token and username/password)
+- Search functionality
+- Resource management (URL, S3, Kafka)
+- Organization management
+- Comprehensive testing suite

ndp_ep-0.1.0/README.md

@@ -0,0 +1,177 @@
+# ndp-ep
+
+[![CI](https://github.com/sci-ndp/ndp-ep-py/workflows/CI/badge.svg)](https://github.com/sci-ndp/ndp-ep-py/actions)
+[![codecov](https://codecov.io/gh/sci-ndp/ndp-ep-py/branch/main/graph/badge.svg)](https://codecov.io/gh/sci-ndp/ndp-ep-py)
+[![PyPI version](https://badge.fury.io/py/ndp-ep.svg)](https://badge.fury.io/py/ndp-ep)
+[![Python versions](https://img.shields.io/pypi/pyversions/ndp-ep.svg)](https://pypi.org/project/ndp-ep/)
+
+A Python client library for interacting with the NDP EP API. This library provides a simple and intuitive interface for managing datasets, organizations, resources, and services through the API.
+
+## Features
+
+- **Complete API Coverage**: Support for all API endpoints including Kafka topics, S3 resources, URL resources, organizations, and services
+- **Authentication**: Token-based and username/password authentication
+- **Search Functionality**: Advanced search capabilities across datasets and resources
+- **Error Handling**: Comprehensive error handling with meaningful error messages
+- **Type Hints**: Full type hint support for better IDE integration
+- **Testing**: Extensive test coverage (>70%) with unit and integration tests
+
+## Installation
+
+```bash
+pip install ndp-ep
+```
+
+## Quick Start
+
+```python
+from ndp_ep import APIClient
+
+# Initialize client with token
+client = APIClient(
+    base_url="https://your-api-endpoint.com",
+    token="your-access-token"
+)
+
+# Or with username/password
+client = APIClient(
+    base_url="https://your-api-endpoint.com",
+    username="your-username",
+    password="your-password"
+)
+
+# List organizations
+organizations = client.list_organizations()
+print(organizations)
+
+# Search datasets
+results = client.search_datasets(
+    terms=["climate", "temperature"],
+    server="global"
+)
+
+# Register a new organization
+org_data = {
+    "name": "my_organization",
+    "title": "My Organization",
+    "description": "A sample organization"
+}
+response = client.register_organization(org_data)
+
+# Register a service
+service_data = {
+    "service_name": "user_auth_api",
+    "service_title": "User Authentication API",
+    "owner_org": "services",
+    "service_url": "https://api.example.com/auth",
+    "service_type": "API",
+    "notes": "RESTful API for user authentication"
+}
+response = client.register_service(service_data)
+```
+
+## API Coverage
+
+### Authentication
+- Token-based authentication
+- Username/password authentication
+
+### Organizations
+- `list_organizations()` - List all organizations
+- `register_organization()` - Create new organization
+- `delete_organization()` - Delete organization
+
+### Datasets and Resources
+- `search_datasets()` - Search datasets with advanced filters
+- `advanced_search()` - Advanced search with POST method
+- `register_url()` - Register URL resources
+- `register_s3_link()` - Register S3 resources
+- `register_kafka_topic()` - Register Kafka topics
+- `register_general_dataset()` - Register general datasets
+- `update_url_resource()` - Update URL resources
+- `update_s3_resource()` - Update S3 resources
+- `update_kafka_topic()` - Update Kafka topics
+- `update_general_dataset()` - Update general datasets (PUT)
+- `patch_general_dataset()` - Partially update general datasets (PATCH)
+- `delete_resource_by_id()` - Delete resource by ID
+- `delete_resource_by_name()` - Delete resource by name
+
+### Services
+- `register_service()` - Register new services
+
+### System Information
+- `get_kafka_details()` - Get Kafka connection details
+- `get_system_status()` - Check system status
+- `get_system_metrics()` - Get system metrics
+- `get_jupyter_details()` - Get Jupyter connection details
+
+## Development
+
+### Setting up development environment
+
+```bash
+# Clone the repository
+git clone https://github.com/sci-ndp/ndp-ep-py.git
+cd ndp-ep-py
+
+# Create virtual environment
+python -m venv venv
+source venv/bin/activate  # On Windows: venv\Scripts\activate
+
+# Install development dependencies
+pip install -e .
+pip install -r requirements-dev.txt
+```
+
+### Running tests
+
+```bash
+# Run all tests
+pytest
+
+# Run with coverage
+pytest --cov=ndp_ep --cov-report=html
+
+# Run specific test categories
+pytest -m unit          # Unit tests only
+pytest -m integration   # Integration tests only
+```
+
+### Code formatting and linting
+
+```bash
+# Format code
+black ndp_ep tests
+
+# Lint code
+flake8 ndp_ep
+
+# Type checking
+mypy ndp_ep
+```
+
+## Contributing
+
+1. Fork the repository
+2. Create a feature branch (`git checkout -b feature/amazing-feature`)
+3. Make your changes
+4. Add tests for your changes
+5. Ensure all tests pass and coverage is maintained
+6. Commit your changes (`git commit -m 'Add amazing feature'`)
+7. Push to the branch (`git push origin feature/amazing-feature`)
+8. Open a Pull Request
+
+## License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+## Changelog
+
+### v0.1.0
+- Initial release
+- Complete API coverage for all endpoints
+- Authentication support (token and username/password)
+- Search functionality
+- Resource management (URL, S3, Kafka)
+- Organization management
+- Comprehensive testing suite

ndp_ep-0.1.0/ndp_ep/__init__.py

@@ -0,0 +1,40 @@
+"""
+NDP EP Python Client Library.
+
+A Python client library for interacting with the NDP EP API.
+This library provides a simple and intuitive interface for managing
+datasets, organizations, resources, and services through the API.
+"""
+
+from .client_base import APIClientBase
+from .search_method import APIClientSearch
+from .register_organization_method import APIClientOrganizationRegister
+from .register_kafka_method import APIClientKafkaRegister
+from .register_s3_method import APIClientS3Register
+from .register_url_method import APIClientURLRegister
+from .register_service_method import APIClientServiceRegister
+from .register_dataset_method import APIClientDatasetRegister
+from .list_organization_method import APIClientOrganizationList
+from .update_kafka_method import APIClientKafkaUpdate
+from .update_s3_method import APIClientS3Update
+from .update_url_method import APIClientURLUpdate
+from .update_dataset_method import APIClientDatasetUpdate
+from .delete_organization_method import APIClientOrganizationDelete
+from .delete_resource_method import APIClientResourceDelete
+from .get_kafka_details_method import APIClientKafkaDetails
+from .get_system_status_method import APIClientSystemStatus
+from .api_client import APIClient
+
+__version__ = "0.1.0"
+__description__ = "Python client library for NDP EP API"
+
+# Main exports
+__all__ = [
+    "APIClient",
+    "APIClientBase",
+    "__version__",
+    "__description__",
+]
+
+# Default client for backward compatibility
+Client = APIClient

ndp_ep-0.1.0/ndp_ep/api_client.py

@@ -0,0 +1,63 @@
+"""Unified API Client combining all functionality."""
+
+from .register_kafka_method import APIClientKafkaRegister
+from .register_organization_method import APIClientOrganizationRegister
+from .register_s3_method import APIClientS3Register
+from .register_url_method import APIClientURLRegister
+from .register_service_method import APIClientServiceRegister
+from .register_dataset_method import APIClientDatasetRegister
+from .list_organization_method import APIClientOrganizationList
+from .search_method import APIClientSearch
+from .update_kafka_method import APIClientKafkaUpdate
+from .update_s3_method import APIClientS3Update
+from .update_url_method import APIClientURLUpdate
+from .update_dataset_method import APIClientDatasetUpdate
+from .delete_organization_method import APIClientOrganizationDelete
+from .delete_resource_method import APIClientResourceDelete
+from .get_kafka_details_method import APIClientKafkaDetails
+from .get_system_status_method import APIClientSystemStatus
+
+
+class APIClient(
+    APIClientKafkaRegister,
+    APIClientOrganizationRegister,
+    APIClientS3Register,
+    APIClientURLRegister,
+    APIClientServiceRegister,
+    APIClientDatasetRegister,
+    APIClientOrganizationList,
+    APIClientSearch,
+    APIClientKafkaUpdate,
+    APIClientS3Update,
+    APIClientURLUpdate,
+    APIClientDatasetUpdate,
+    APIClientOrganizationDelete,
+    APIClientResourceDelete,
+    APIClientKafkaDetails,
+    APIClientSystemStatus,
+):
+    """
+    Unified API Client with comprehensive functionality.
+
+    This class combines all the individual API client mixins to provide
+    a complete interface for interacting with the NDP EP API.
+
+    Features:
+    - Organization management (create, list, delete)
+    - Resource registration (Kafka, S3, URL, Services, General datasets)
+    - Resource updates (Kafka, S3, URL, General datasets)
+    - Resource deletion (by ID and name)
+    - Search functionality (simple and advanced)
+    - System information (status, metrics, Kafka details, Jupyter details)
+    - Authentication (token-based and username/password)
+
+    Example:
+        >>> client = APIClient(
+        ...     base_url="https://api.example.com",
+        ...     token="your-token"
+        ... )
+        >>> organizations = client.list_organizations()
+        >>> results = client.search_datasets(["climate"], server="global")
+    """
+
+    pass

ndp_ep-0.1.0/ndp_ep/client_base.py

@@ -0,0 +1,147 @@
+"""Base class for the API client."""
+
+import requests
+from typing import Optional
+from urllib.parse import urlparse
+
+
+class APIClientBase:
+    """Base class for the API client."""
+
+    def __init__(
+        self,
+        base_url: str,
+        token: Optional[str] = None,
+        username: Optional[str] = None,
+        password: Optional[str] = None,
+    ) -> None:
+        """
+        Initialize the API client.
+
+        Args:
+            base_url: Base URL of the API.
+            token: Access token for authentication.
+            username: Username for authentication.
+            password: Password for authentication.
+
+        Raises:
+            ValueError: If invalid authentication combination is provided
+                       or if API is not reachable.
+        """
+        self.base_url = self._ensure_protocol(base_url).rstrip("/")
+        self.session = requests.Session()
+
+        # Initialize token to None by default
+        self.token: Optional[str] = None
+
+        # Validate input combinations
+        if token and (username or password):
+            raise ValueError(
+                "Provide either a token or username/password, not both."
+            )
+
+        # Initialize with token if provided
+        if token:
+            self.token = token
+            self.session.headers.update(
+                {"Authorization": f"Bearer {self.token}"}
+            )
+        # Fallback to username/password authentication
+        elif username and password:
+            self.get_token(username, password)
+        # Check API availability if no authentication details are provided
+        else:
+            self._check_api_availability()
+
+    @staticmethod
+    def _ensure_protocol(url: str) -> str:
+        """
+        Ensure the URL contains a valid protocol.
+
+        If missing, prepend 'http://'.
+
+        Args:
+            url: The URL to validate.
+
+        Returns:
+            The URL with a protocol.
+        """
+        parsed_url = urlparse(url)
+        if not parsed_url.scheme:
+            return f"http://{url}"
+        return url
+
+    def _check_api_availability(self) -> None:
+        """
+        Check if the API is reachable.
+
+        Makes a GET request to the base URL.
+
+        Raises:
+            ValueError: If the connection fails or response is not 200.
+        """
+        try:
+            response = self.session.get(self.base_url)
+            response.raise_for_status()
+        except requests.exceptions.ConnectionError:
+            raise ValueError(
+                f"Failed to connect to the API at {self.base_url}. "
+                "Please check if the URL is correct and reachable."
+            )
+        except requests.exceptions.HTTPError as http_err:
+            raise ValueError(
+                "API connection check failed with "
+                f"status code {response.status_code}: {http_err}"
+            )
+        except requests.exceptions.RequestException as req_err:
+            raise ValueError(
+                "An error occurred while attempting to connect "
+                f"to the API: {req_err}"
+            )
+
+    def get_token(self, username: str, password: str) -> None:
+        """
+        Obtain authentication token.
+
+        Args:
+            username: Username for authentication.
+            password: Password for authentication.
+
+        Raises:
+            ValueError: If authentication fails or connection error occurs.
+        """
+        url = f"{self.base_url}/token"
+        try:
+            response = self.session.post(
+                url, data={"username": username, "password": password}
+            )
+            response.raise_for_status()
+            token_data = response.json()
+            self.token = token_data.get("access_token")
+            if not self.token:
+                raise ValueError(
+                    "Authentication failed: No access token received."
+                )
+            # Update session headers with the token
+            self.session.headers.update(
+                {"Authorization": f"Bearer {self.token}"}
+            )
+        except requests.exceptions.ConnectionError:
+            raise ValueError(
+                f"Failed to connect to the API at {self.base_url}. "
+                "Please check if the URL is correct and reachable."
+            )
+        except requests.exceptions.HTTPError as http_err:
+            if response.status_code == 401:
+                raise ValueError(
+                    "Authentication failed: Invalid username or password."
+                ) from http_err
+            else:
+                raise ValueError(
+                    f"HTTP error occurred: {http_err}"
+                ) from http_err
+        except requests.exceptions.RequestException as req_err:
+            raise ValueError(
+                "An error occurred while attempting to obtain "
+                f"the token: {req_err}"
+            ) from req_err