rara-tools 0.0.1__tar.gz → 0.0.2__tar.gz

rara_tools-0.0.2/PKG-INFO

@@ -0,0 +1,175 @@
+Metadata-Version: 2.1
+Name: rara-tools
+Version: 0.0.2
+Summary: Tools to support Kata's work.
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Intended Audience :: Science/Research
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE.md
+Requires-Dist: elasticsearch==8.*
+Requires-Dist: elasticsearch_dsl==8.*
+Requires-Dist: minio==7.*
+Requires-Dist: requests
+Provides-Extra: testing
+Requires-Dist: pytest>=8.0; extra == "testing"
+Requires-Dist: pytest-order; extra == "testing"
+
+# RaRa Tools
+
+![Py3.10](https://img.shields.io/badge/python-3.10-green.svg)
+![Py3.11](https://img.shields.io/badge/python-3.11-green.svg)
+![Py3.12](https://img.shields.io/badge/python-3.12-green.svg)
+
+**`rara-tools`** is a Python library containing various generic tools used by components of Kata.
+
+---
+
+## ✨ Features  
+
+- Elasticsearch index & document operations
+- S3 file management operations
+- Task reporting to Core API
+---
+
+## ⚡ Quick Start  
+
+Get started with `rara-tools` in just a few steps:
+
+1. **Install the Package**  
+   Ensure you're using Python 3.10 or above, then run:  
+   ```bash
+   pip install rara-tools
+   ```
+
+2. **Import and Use**  
+   Example usage to download a folder from S3:  
+
+   ```python
+   from rara_tools.s3 import S3Files
+
+   s3 = S3Files(
+        url="your-s3-address",
+        access_key = "xxx",
+        secret_key = "yyy",
+        bucket = "my-sad-bucket"
+   )
+
+   s3.download("my-folder-in-s3")
+   ```
+
+---
+
+## ⚙️ Installation Guide
+
+Follow the steps below to install the `rara-tools` package, either via `pip` or locally.
+
+---
+
+### Installation via `pip`
+
+<details><summary>Click to expand</summary>
+
+1. **Set Up Your Python Environment**  
+   Create or activate a Python environment using Python **3.10** or above.
+
+2. **Install the Package**  
+   Run the following command:  
+   ```bash
+   pip install rara-tools
+   ```
+</details>
+
+---
+
+### Local Installation
+
+Follow these steps to install the `rara-tools` package locally:  
+
+<details><summary>Click to expand</summary>
+
+1. **Clone the Repository**  
+   Clone the repository and navigate into it:  
+   ```bash
+   git clone <repository-url>
+   cd <repository-directory>
+   ```
+
+2. **Set Up Python Environment**  
+   Create or activate a Python environment using Python 3.10 or above. E.g:
+   ```bash
+   conda create -n py310 python==3.10
+   conda activate py310
+   ```
+
+3. **Install Build Package**  
+   Install the `build` package to enable local builds:  
+   ```bash
+   pip install build
+   ```
+
+4. **Build the Package**  
+   Run the following command inside the repository:  
+   ```bash
+   python -m build
+   ```
+
+5. **Install the Package**  
+   Install the built package locally:  
+   ```bash
+   pip install .
+   ```
+
+</details>
+
+---
+
+## 🚀 Testing Guide
+
+Follow these steps to test the `rara-tools` package.
+
+
+### How to Test
+
+<details><summary>Click to expand</summary>
+
+1. **Clone the Repository**  
+   Clone the repository and navigate into it:  
+   ```bash
+   git clone <repository-url>
+   cd <repository-directory>
+   ```
+
+2. **Set Up Python Environment**  
+   Create or activate a Python environment using Python 3.10 or above.
+
+3. **Install Build Package**  
+   Install the `build` package:  
+   ```bash
+   pip install build
+   ```
+
+4. **Build the Package**  
+   Build the package inside the repository:  
+   ```bash
+   python -m build
+   ```
+
+5. **Install with Testing Dependencies**  
+   Install the package along with its testing dependencies:  
+   ```bash
+   pip install .[testing]
+   ```
+
+6. **Run Tests**  
+   Run the test suite from the repository root:  
+   ```bash
+   python -m pytest -v tests
+   ```
+
+---
+
+</details>

rara_tools-0.0.2/README.md

@@ -0,0 +1,155 @@
+# RaRa Tools
+
+![Py3.10](https://img.shields.io/badge/python-3.10-green.svg)
+![Py3.11](https://img.shields.io/badge/python-3.11-green.svg)
+![Py3.12](https://img.shields.io/badge/python-3.12-green.svg)
+
+**`rara-tools`** is a Python library containing various generic tools used by components of Kata.
+
+---
+
+## ✨ Features  
+
+- Elasticsearch index & document operations
+- S3 file management operations
+- Task reporting to Core API
+---
+
+## ⚡ Quick Start  
+
+Get started with `rara-tools` in just a few steps:
+
+1. **Install the Package**  
+   Ensure you're using Python 3.10 or above, then run:  
+   ```bash
+   pip install rara-tools
+   ```
+
+2. **Import and Use**  
+   Example usage to download a folder from S3:  
+
+   ```python
+   from rara_tools.s3 import S3Files
+
+   s3 = S3Files(
+        url="your-s3-address",
+        access_key = "xxx",
+        secret_key = "yyy",
+        bucket = "my-sad-bucket"
+   )
+
+   s3.download("my-folder-in-s3")
+   ```
+
+---
+
+## ⚙️ Installation Guide
+
+Follow the steps below to install the `rara-tools` package, either via `pip` or locally.
+
+---
+
+### Installation via `pip`
+
+<details><summary>Click to expand</summary>
+
+1. **Set Up Your Python Environment**  
+   Create or activate a Python environment using Python **3.10** or above.
+
+2. **Install the Package**  
+   Run the following command:  
+   ```bash
+   pip install rara-tools
+   ```
+</details>
+
+---
+
+### Local Installation
+
+Follow these steps to install the `rara-tools` package locally:  
+
+<details><summary>Click to expand</summary>
+
+1. **Clone the Repository**  
+   Clone the repository and navigate into it:  
+   ```bash
+   git clone <repository-url>
+   cd <repository-directory>
+   ```
+
+2. **Set Up Python Environment**  
+   Create or activate a Python environment using Python 3.10 or above. E.g:
+   ```bash
+   conda create -n py310 python==3.10
+   conda activate py310
+   ```
+
+3. **Install Build Package**  
+   Install the `build` package to enable local builds:  
+   ```bash
+   pip install build
+   ```
+
+4. **Build the Package**  
+   Run the following command inside the repository:  
+   ```bash
+   python -m build
+   ```
+
+5. **Install the Package**  
+   Install the built package locally:  
+   ```bash
+   pip install .
+   ```
+
+</details>
+
+---
+
+## 🚀 Testing Guide
+
+Follow these steps to test the `rara-tools` package.
+
+
+### How to Test
+
+<details><summary>Click to expand</summary>
+
+1. **Clone the Repository**  
+   Clone the repository and navigate into it:  
+   ```bash
+   git clone <repository-url>
+   cd <repository-directory>
+   ```
+
+2. **Set Up Python Environment**  
+   Create or activate a Python environment using Python 3.10 or above.
+
+3. **Install Build Package**  
+   Install the `build` package:  
+   ```bash
+   pip install build
+   ```
+
+4. **Build the Package**  
+   Build the package inside the repository:  
+   ```bash
+   python -m build
+   ```
+
+5. **Install with Testing Dependencies**  
+   Install the package along with its testing dependencies:  
+   ```bash
+   pip install .[testing]
+   ```
+
+6. **Run Tests**  
+   Run the test suite from the repository root:  
+   ```bash
+   python -m pytest -v tests
+   ```
+
+---
+
+</details>

rara_tools-0.0.2/VERSION

@@ -0,0 +1 @@
+0.0.2

{rara_tools-0.0.1 → rara_tools-0.0.2}/rara_tools/elastic.py

@@ -8,7 +8,7 @@ from .decorators import _elastic_connection
 class KataElastic:
     """A class to manage all required Elasticsearch operations for Kata.
     """
-    def __init__(self, elasticsearch_url, timeout: Optional[int] = None):
+    def __init__(self, elasticsearch_url: str, timeout: Optional[int] = None):
         self.timeout = timeout
         self.elasticsearch_url = elasticsearch_url
         self.elasticsearch = Elasticsearch(self.elasticsearch_url, request_timeout=self.timeout)
@@ -16,7 +16,7 @@ class KataElastic:
     @_elastic_connection
     def check(self) -> bool:
         """Checks Elasticsearch connection.
-        :return: bool: Elasticsearch accessible or not.
+        :return: bool: Elasticsearch alive or dead.
         """
         if self.elasticsearch.ping():
             return True
@@ -31,7 +31,7 @@ class KataElastic:
             settings: Optional[dict] = None
     ) -> Dict:
         """Creates empty index.
-        :param: index str: Name of the created index.
+        :param: index str: Name of the index to create.
         :param: shards int: Number of shards for the index.
         :param: replicas int: Number of replicas of the index.
         :param: settings dict: Overwrite settings for the index.
@@ -43,22 +43,24 @@ class KataElastic:
         return self.elasticsearch.indices.create(index=index, settings=body)
 
     @_elastic_connection
-    def delete_index(self, index: str, ignore: Optional[bool] = True) -> dict:
+    def delete_index(self, index: str, ignore: Optional[bool] = True) -> Dict:
         """Deletes index.
         :param: index str: Name of the index to be deleted.
         :param: ignore bool: Ignore errors because of closed/deleted index.
+        :return: Dict of Elastic's acknowledgement of the action.
         """
         response = self.elasticsearch.indices.delete(index=index, ignore_unavailable=ignore)
         return response
 
     @_elastic_connection
-    def delete_document(self, document_id: str, index: str, ignore: Optional[bool] = True) -> dict:
+    def delete_document(self, index: str, document_id: str) -> Dict:
         """Deletes document fom index.
         :param: document_id str: ID of the document to be deleted.
         :param: index str: Index where the document is to be found.
         :param: ignore bool: Ignore errors because of closed/deleted index.
+        :return: Dict of Elastic's acknowledgement of the action.
         """
-        response = self.elasticsearch.delete(id=document_id, index=index, ignore_unavailable=ignore)
+        response = self.elasticsearch.delete(id=document_id, index=index)
         return response
 
     @_elastic_connection
@@ -67,12 +69,13 @@ class KataElastic:
         :param: index str: Index that document will be indexed into.
         :param: body dict: Document body.
         :param: document_id str: Optional id for the document. Is generated automatically if None.
+        :return: Dict of Elastic's acknowledgement of the action.
         """
         if document_id:
-            document = self.elasticsearch.index(index=index, id=document_id, body=body)
+            indexed = self.elasticsearch.index(index=index, id=document_id, body=body)
         else:
-            document = self.elasticsearch.index(index=index, body=body)
-        return document
+            indexed = self.elasticsearch.index(index=index, body=body)
+        return indexed
 
     @_elastic_connection
     def get_documents_by_key(self, index: str, document_key: str) -> List:

{rara_tools-0.0.1 → rara_tools-0.0.2}/rara_tools/s3.py

@@ -1,6 +1,6 @@
 import os
 import uuid
-from typing import Optional
+from typing import Optional, List, Generator
 from minio import Minio
 
 from .exceptions import S3InitException, S3ConnectionException, S3InputException
@@ -42,7 +42,7 @@ class S3Files:
             raise S3InputException(f"File '{file_path}' does not exist in file system!")
         return self.minio_client.fput_object(self.bucket, s3_path_name, file_path)
 
-    def list(self, prefix: Optional[str] = ""):
+    def list(self, prefix: Optional[str] = "") -> List:
         """Lists all available files in S3 bucket.
         :param: prefix str: Limits the listing to a given prefix.
         :return: List of file paths in S3.
@@ -51,16 +51,18 @@ class S3Files:
         list_of_objects = [o.object_name for o in list_of_objects]
         return list_of_objects
 
-    def delete(self, path: str):
+    def delete(self, path: str) -> bool:
         """Deletes file in S3.
         :param: path str: Path of the file in S3 to be deleted.
+        :return: True.
         """
         list_of_objects = self.minio_client.list_objects(self.bucket, prefix=path, recursive=True)
         list_of_objects = [o.object_name for o in list_of_objects]
         for path_to_delete in list_of_objects:
             self.minio_client.remove_object(self.bucket, path_to_delete)
+        return True
 
-    def download(self, path: str, download_dir: Optional[str] = "."):
+    def download(self, path: str, download_dir: Optional[str] = ".") -> Generator[str, str, str]:
         """Downloads file or folder from S3.
         :param: path str: Path to the file or folder in S3.
         :param: download_dir str: Directory to download the files and folders into.
@@ -72,26 +74,26 @@ class S3Files:
             self.minio_client.fget_object(self.bucket, minio_object.object_name, full_path)
             yield full_path
 
-    def upload(self, path: str, prefix: Optional[str] = ""):
+    def upload(self, path: str, prefix: Optional[str] = "") -> str:
         """Uploads file or folder to S3 bucket.
         :param: path str: Path to the file to upload in local file system.
         :param: prefix str: Optional prefix for S3 path.
         :returns: File path of the uploaded file or folder in S3.
         """
-        path_name = os.path.split(path)[-1]
         # Manage directories
         if os.path.isdir(path):
             s3_path_prefix = f"{prefix}{uuid.uuid4().hex}"
             for root, _, files in os.walk(path):
                 for file in files:
+                    relative_prefix = root.removeprefix(path).lstrip("/")
                     file_path = os.path.join(root, file)
                     extension = file.split(".")[-1]
-                    s3_file_name = f"{uuid.uuid4().hex}.{extension}"
-                    s3_path_name = os.path.join(s3_path_prefix, s3_file_name)
+                    s3_path_name = os.path.join(s3_path_prefix, relative_prefix, file)
                     self._put_file(file_path, s3_path_name)
             return s3_path_prefix
         # Manage single files
         else:
+            path_name = os.path.split(path)[-1]
             extension = path_name.split(".")[-1]
             s3_path_name = f"{prefix}{uuid.uuid4().hex}.{extension}"
             self._put_file(path, s3_path_name)

rara_tools-0.0.2/rara_tools.egg-info/PKG-INFO

@@ -0,0 +1,175 @@
+Metadata-Version: 2.1
+Name: rara-tools
+Version: 0.0.2
+Summary: Tools to support Kata's work.
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Intended Audience :: Science/Research
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE.md
+Requires-Dist: elasticsearch==8.*
+Requires-Dist: elasticsearch_dsl==8.*
+Requires-Dist: minio==7.*
+Requires-Dist: requests
+Provides-Extra: testing
+Requires-Dist: pytest>=8.0; extra == "testing"
+Requires-Dist: pytest-order; extra == "testing"
+
+# RaRa Tools
+
+![Py3.10](https://img.shields.io/badge/python-3.10-green.svg)
+![Py3.11](https://img.shields.io/badge/python-3.11-green.svg)
+![Py3.12](https://img.shields.io/badge/python-3.12-green.svg)
+
+**`rara-tools`** is a Python library containing various generic tools used by components of Kata.
+
+---
+
+## ✨ Features  
+
+- Elasticsearch index & document operations
+- S3 file management operations
+- Task reporting to Core API
+---
+
+## ⚡ Quick Start  
+
+Get started with `rara-tools` in just a few steps:
+
+1. **Install the Package**  
+   Ensure you're using Python 3.10 or above, then run:  
+   ```bash
+   pip install rara-tools
+   ```
+
+2. **Import and Use**  
+   Example usage to download a folder from S3:  
+
+   ```python
+   from rara_tools.s3 import S3Files
+
+   s3 = S3Files(
+        url="your-s3-address",
+        access_key = "xxx",
+        secret_key = "yyy",
+        bucket = "my-sad-bucket"
+   )
+
+   s3.download("my-folder-in-s3")
+   ```
+
+---
+
+## ⚙️ Installation Guide
+
+Follow the steps below to install the `rara-tools` package, either via `pip` or locally.
+
+---
+
+### Installation via `pip`
+
+<details><summary>Click to expand</summary>
+
+1. **Set Up Your Python Environment**  
+   Create or activate a Python environment using Python **3.10** or above.
+
+2. **Install the Package**  
+   Run the following command:  
+   ```bash
+   pip install rara-tools
+   ```
+</details>
+
+---
+
+### Local Installation
+
+Follow these steps to install the `rara-tools` package locally:  
+
+<details><summary>Click to expand</summary>
+
+1. **Clone the Repository**  
+   Clone the repository and navigate into it:  
+   ```bash
+   git clone <repository-url>
+   cd <repository-directory>
+   ```
+
+2. **Set Up Python Environment**  
+   Create or activate a Python environment using Python 3.10 or above. E.g:
+   ```bash
+   conda create -n py310 python==3.10
+   conda activate py310
+   ```
+
+3. **Install Build Package**  
+   Install the `build` package to enable local builds:  
+   ```bash
+   pip install build
+   ```
+
+4. **Build the Package**  
+   Run the following command inside the repository:  
+   ```bash
+   python -m build
+   ```
+
+5. **Install the Package**  
+   Install the built package locally:  
+   ```bash
+   pip install .
+   ```
+
+</details>
+
+---
+
+## 🚀 Testing Guide
+
+Follow these steps to test the `rara-tools` package.
+
+
+### How to Test
+
+<details><summary>Click to expand</summary>
+
+1. **Clone the Repository**  
+   Clone the repository and navigate into it:  
+   ```bash
+   git clone <repository-url>
+   cd <repository-directory>
+   ```
+
+2. **Set Up Python Environment**  
+   Create or activate a Python environment using Python 3.10 or above.
+
+3. **Install Build Package**  
+   Install the `build` package:  
+   ```bash
+   pip install build
+   ```
+
+4. **Build the Package**  
+   Build the package inside the repository:  
+   ```bash
+   python -m build
+   ```
+
+5. **Install with Testing Dependencies**  
+   Install the package along with its testing dependencies:  
+   ```bash
+   pip install .[testing]
+   ```
+
+6. **Run Tests**  
+   Run the test suite from the repository root:  
+   ```bash
+   python -m pytest -v tests
+   ```
+
+---
+
+</details>

{rara_tools-0.0.1 → rara_tools-0.0.2}/tests/test_elastic.py

@@ -12,6 +12,7 @@ es_url = os.getenv("ELASTIC_TEST_URL", "http://localhost:9200")
 ELASTIC = KataElastic(es_url)
 ELASTIC_BAD = KataElastic("http://locallost:9012")
 TEST_INDEX_NAME = "tools_testing_index"
+TEST_DOCUMENT_ID = None
 
 
 @pytest.mark.order(1)
@@ -43,11 +44,24 @@ def test_get_document_by_key():
     result = ELASTIC.get_documents_by_key(TEST_INDEX_NAME, "foo")
     assert len(result) == 2
     result = ELASTIC.get_documents_by_key(TEST_INDEX_NAME, "bar")
+    global TEST_DOCUMENT_ID
+    TEST_DOCUMENT_ID = result[0].meta.id
     assert len(result) == 1
     result = ELASTIC.get_documents_by_key(TEST_INDEX_NAME, "loll")
     assert len(result) == 0
 
-@pytest.mark.order(4)
+@pytest.mark.order(5)
+def test_document_deleting():
+    """ Tests deleting a document from index.
+    """
+    deleted = ELASTIC.delete_document(TEST_INDEX_NAME, TEST_DOCUMENT_ID)
+    assert deleted["result"] == "deleted"
+    sleep(1)
+    # check if document was actually deleted
+    result = ELASTIC.get_documents_by_key(TEST_INDEX_NAME, "bar")
+    assert len(result) == 0
+
+@pytest.mark.order(5)
 def test_index_deleting():
     """ Tests deleting index. We delete the test index now.
     """

rara_tools-0.0.1/PKG-INFO

@@ -1,21 +0,0 @@
-Metadata-Version: 2.1
-Name: rara-tools
-Version: 0.0.1
-Summary: Tools to support Kata's work.
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Intended Audience :: Science/Research
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-License-File: LICENSE.md
-Requires-Dist: elasticsearch==8.*
-Requires-Dist: elasticsearch_dsl==8.*
-Requires-Dist: minio==7.*
-Requires-Dist: requests
-Provides-Extra: testing
-Requires-Dist: pytest>=8.0; extra == "testing"
-Requires-Dist: pytest-order; extra == "testing"
-
-# rara-tools

rara_tools-0.0.1/README.md

@@ -1 +0,0 @@
-# rara-tools

rara_tools-0.0.1/VERSION

@@ -1 +0,0 @@
-0.0.1

rara_tools-0.0.1/rara_tools.egg-info/PKG-INFO

@@ -1,21 +0,0 @@
-Metadata-Version: 2.1
-Name: rara-tools
-Version: 0.0.1
-Summary: Tools to support Kata's work.
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Intended Audience :: Science/Research
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-License-File: LICENSE.md
-Requires-Dist: elasticsearch==8.*
-Requires-Dist: elasticsearch_dsl==8.*
-Requires-Dist: minio==7.*
-Requires-Dist: requests
-Provides-Extra: testing
-Requires-Dist: pytest>=8.0; extra == "testing"
-Requires-Dist: pytest-order; extra == "testing"
-
-# rara-tools