oceanprotocol-job-details 0.2.4__py3-none-any.whl → 0.2.6__py3-none-any.whl

oceanprotocol_job_details/ocean.py

@@ -5,7 +5,17 @@ import os
 from dataclasses import dataclass, field
 from functools import cached_property
 from pathlib import Path
-from typing import Any, Generic, Iterator, Optional, Sequence, Type, TypeVar, final
+from typing import (
+    Any,
+    Generator,
+    Generic,
+    Iterator,
+    Optional,
+    Sequence,
+    Type,
+    TypeVar,
+    final,
+)
 
 import orjson
 from dataclasses_json import config as dc_config
@@ -244,6 +254,11 @@ class JobDetails(Generic[T]):
         if not hasattr(self._type, "__dataclass_fields__"):
             raise TypeError(f"{self._type} is not a dataclass type")
 
+    def next_path(self) -> Generator[tuple[int, Path], None, None]:
+        for idx, did_files in enumerate(self.files):
+            for file in did_files.input_files:
+                yield (idx, file)
+
     @cached_property
     def input_parameters(self) -> T:
         """Read the input parameters and return them in an instance of the dataclass T"""

oceanprotocol_job_details-0.2.6.dist-info/METADATA

@@ -0,0 +1,125 @@
+Metadata-Version: 2.4
+Name: oceanprotocol-job-details
+Version: 0.2.6
+Summary: A Python package to get details from OceanProtocol jobs
+Project-URL: Homepage, https://github.com/AgrospAI/oceanprotocol-job-details
+Project-URL: Issues, https://github.com/AgrospAI/oceanprotocol-job-details/issues
+Author-email: Agrospai <agrospai@udl.cat>, Christian López García <christian.lopez@udl.cat>
+License: Copyright 2025 Agrospai
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+License-File: LICENSE
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.10
+Requires-Dist: dataclasses-json>=0.6.7
+Requires-Dist: dependency-injector>=4.48.2
+Requires-Dist: orjson>=3.11.3
+Description-Content-Type: text/markdown
+
+A Python package to get details from OceanProtocol jobs
+
+---
+
+## Installation
+
+```
+pip install oceanprotocol-job-details
+```
+
+## Usage 
+
+As a simple library, we only need to import `JobDetails` and load it, it will:
+
+1. Fetch the needed parameters to populate the `JobDetails` instance from the environment variables or use the passed values to the `load()` method.
+1. Look for the files corresponding to the passed DIDs in the filesystem according to the [Ocean Protocol Structure](#oceanprotocol-structure) and load them into the `JobDetails` instance.
+
+
+### Minimal Example
+
+```python
+from oceanprotocol_job_details import JobDetails
+
+job_details = JobDetails.load()
+```
+
+### Custom Input Parameters
+
+If our algorithm has custom input parameters and we want to load them into our algorithm, we can do it as follows:
+
+```python
+from dataclasses import dataclass
+from oceanprotocol_job_details import JobDetails
+
+
+@dataclass
+class InputParameters:
+    foobar: str
+
+
+job_details = JobDetails[InputParameters].load(InputParameters)
+
+# Usage
+job_details.input_parameters.foobar
+```
+
+```python
+from dataclasses import dataclass
+from oceanprotocol_job_details import JobDetails
+
+
+@dataclass
+class Foo:
+    bar: str
+
+
+@dataclass
+class InputParameters:
+    # Allows for nested types
+    foo: Foo
+
+
+job_details = JobDetails[InputParameters].load(InputParameters)
+
+# Usage
+job_details.input_parameters.foo.bar
+```
+
+The values to fill the custom `InputParameters` will be parsed from the `algoCustomData.json` located next to the input data directories. 
+
+### Iterating Input Files the clean way
+
+```python
+from oceanprotocol_job_details import JobDetails
+
+
+job_details = JobDetails.load()
+
+for idx, file_path in job_details.next_file():
+    ...
+
+# Or if you just want one file path
+_, file_path = job_details.next_file()
+```
+
+## OceanProtocol Structure
+
+```bash
+data        # Root /data directory
+├── ddos    # Contains the loaded dataset's DDO
+│   ├── 17feb...e42 # DDO file
+│   └── ... # One DDO per loaded dataset
+├── inputs  # Datasets dir
+│   ├── 17feb...e42 # Dir holding the data of its name DID, contains files named 0..X
+│   │   └── 0 # Data file
+│   └── algoCustomData.json # Custom algorithm input data
+├── logs    # Algorithm output logs dir
+└── outputs # Algorithm output files dir
+```
+
+> **_Note:_** Even though it's possible that the algorithm is passed multiple datasets, right now the implementation only allows to use **one dataset** per algorithm execution, so **normally** the executing job will only have **one ddo**, **one dir** inside inputs, and **one data file** named `0`.

{oceanprotocol_job_details-0.2.4.dist-info → oceanprotocol_job_details-0.2.6.dist-info}/RECORD

@@ -1,6 +1,6 @@
 oceanprotocol_job_details/__init__.py,sha256=C67wv7fy5ZT5FtbGD-oQeSoLU-x6e2ts-820koFo034,55
 oceanprotocol_job_details/di.py,sha256=j2BvpaGpiQwt9MSiViYPFup4MnzTHm44Zq12_TQ07kY,1180
-oceanprotocol_job_details/ocean.py,sha256=-LdztJ2h1WS9OVWVzc8jAfLcxhzJ9Zqsn8Rk0d7SZOo,7118
+oceanprotocol_job_details/ocean.py,sha256=uABP0JZdCpd3NrTmfXWGuG8XOEVx680-jjRDV0hHync,7373
 oceanprotocol_job_details/paths.py,sha256=N90IF8OQCBELULGPaBv9yALApa-lC2dsVKXMWLdqa14,851
 oceanprotocol_job_details/loaders/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 oceanprotocol_job_details/loaders/loader.py,sha256=HIzsVKCuGP7ghfM7ppN3ANVybvsA64wr3h8I68mqS6A,195
@@ -8,7 +8,7 @@ oceanprotocol_job_details/loaders/impl/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JC
 oceanprotocol_job_details/loaders/impl/ddo.py,sha256=_xl0PozuvIm0n6jU--Znk6qfw1A6OLBq3SERbDGIF74,844
 oceanprotocol_job_details/loaders/impl/files.py,sha256=oFkA_0Ma5NBgWvVEk_rhyDIDrAam_zjesh0-bxZaIU8,1443
 oceanprotocol_job_details/loaders/impl/job_details.py,sha256=wf0xNAG4tESq57vqkdtMQ8BdiyS91j5f7FL8Gfwbjh4,770
-oceanprotocol_job_details-0.2.4.dist-info/METADATA,sha256=_NdQh6JKQb8debVX1HA_Damd91vuXSOLdazsxqqVsGM,2948
-oceanprotocol_job_details-0.2.4.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
-oceanprotocol_job_details-0.2.4.dist-info/licenses/LICENSE,sha256=ni3ix7P_GxK1W3VGC4fJ3o6QoCngCEpSuTJwO4nkpbw,1055
-oceanprotocol_job_details-0.2.4.dist-info/RECORD,,
+oceanprotocol_job_details-0.2.6.dist-info/METADATA,sha256=jqQAuRh8p34G3ZIUC23-rfr8Odrw9ByyW5YP5FUyt9g,4507
+oceanprotocol_job_details-0.2.6.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
+oceanprotocol_job_details-0.2.6.dist-info/licenses/LICENSE,sha256=ni3ix7P_GxK1W3VGC4fJ3o6QoCngCEpSuTJwO4nkpbw,1055
+oceanprotocol_job_details-0.2.6.dist-info/RECORD,,

oceanprotocol_job_details-0.2.4.dist-info/METADATA

@@ -1,76 +0,0 @@
-Metadata-Version: 2.4
-Name: oceanprotocol-job-details
-Version: 0.2.4
-Summary: A Python package to get details from OceanProtocol jobs
-Project-URL: Homepage, https://github.com/AgrospAI/oceanprotocol-job-details
-Project-URL: Issues, https://github.com/AgrospAI/oceanprotocol-job-details/issues
-Author-email: Christian López García <christian.lopez@udl.cat>
-License: Copyright 2025 Agrospai
-        
-        Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
-        
-        The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
-        
-        THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
-License-File: LICENSE
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Operating System :: OS Independent
-Classifier: Programming Language :: Python :: 3
-Requires-Python: >=3.10
-Requires-Dist: dataclasses-json>=0.6.7
-Requires-Dist: dependency-injector>=4.48.2
-Requires-Dist: orjson>=3.11.3
-Description-Content-Type: text/markdown
-
-A Python package to get details from OceanProtocol jobs
-
----
-
-## Installation
-
-```
-pip install oceanprotocol-job-details
-```
-
-## Usage 
-
-As a simple library, we only need to import the main object and use it once:
-
-```Python
-from oceanprotocol_job_details import JobDetails
-
-# Having no algorithm input parameters
-job_details = JobDetails.load()
-
-```
-
-If our algorithm has custom input parameters and we want to load them into our algorithm, we can do it as follows:
-
-```Python
-
-from dataclasses import dataclass
-from oceanprotocol_job_details import JobDetails
-
-
-@dataclass
-class InputParameters:
-    name: str
-    age: int
-
-
-job_details: JobDetails[InputParameters] = JobDetails.load(InputParameters)
-
-# Usage (is type hinted)
-job_details.input_parameters.name
-job_details.input_parameters.age
-
-```
-
-Assumes the directory structure of OceanProtocol algorithms.
-
-### Core functionalities
-
-Given the Ocean Protocol job details structure, parses the passed algorithm parameters into an object to use in your algorithms.
-
-1. Input parameter JSON parsing and validation
-1. Metadata and service extraction from the directory structure.