fractal-task-tools 0.0.2__tar.gz

fractal_task_tools-0.0.2/LICENSE

@@ -0,0 +1,29 @@
+BSD 3-Clause License
+
+Copyright 2024 (C) Friedrich Miescher Institute for Biomedical Research and University of Zurich
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

fractal_task_tools-0.0.2/PKG-INFO

@@ -0,0 +1,85 @@
+Metadata-Version: 2.2
+Name: fractal-task-tools
+Version: 0.0.2
+Summary: Shared tools for Fractal tasks
+Author-email: Tommaso Comparin <tommaso.comparin@exact-lab.it>
+License: BSD-3-Clause
+Project-URL: homepage, https://github.com/fractal-analytics-platform/fractal-task-tools
+Project-URL: repository, https://github.com/fractal-analytics-platform/fractal-task-tools
+Project-URL: changelog, https://github.com/fractal-analytics-platform/fractal-task-tools/blob/main/CHANGELOG.md
+Requires-Python: <3.13,>=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pydantic<=2.8.2,>=2.0.0
+Requires-Dist: docstring-parser==0.15
+Provides-Extra: dev
+Requires-Dist: bumpver==2024.1130; extra == "dev"
+Requires-Dist: devtools==0.12.2; extra == "dev"
+Requires-Dist: pytest<9.0.0,>=8.3.0; extra == "dev"
+Requires-Dist: coverage<7.7.0,>=7.6.0; extra == "dev"
+
+# Fractal task tools
+
+[![PyPI version](https://img.shields.io/pypi/v/fractal-task-tools?color=gree)](https://pypi.org/project/fractal-task-tools/)
+[![License](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)
+[![CI Status](https://github.com/fractal-analytics-platform/fractal-task-tools/actions/workflows/ci.yml/badge.svg)](https://github.com/fractal-analytics-platform/fractal-task-tools/actions/workflows/ci_pip.yml)
+[![Coverage](https://raw.githubusercontent.com/fractal-analytics-platform/fractal-task-tools/python-coverage-comment-action-data/badge.svg)](https://htmlpreview.github.io/?https://github.com/fractal-analytics-platform/fractal-task-tools/blob/python-coverage-comment-action-data/htmlcov/index.html)
+
+Fractal-task-tools provides some basic tools for building tasks for the [Fractal](https://fractal-analytics-platform.github.io/) framework.
+
+![Fractal_overview_small](https://github.com/user-attachments/assets/666c8797-2594-4b8e-b1d2-b43fca66d1df)
+
+[Fractal](https://fractal-analytics-platform.github.io/) is a framework developed at the [BioVisionCenter](https://www.biovisioncenter.uzh.ch/en.html) to process bioimaging data at scale in the OME-Zarr format and prepare the images for interactive visualization.
+
+
+# Get started
+```console
+$ python -m venv venv
+
+$ source venv/bin/activate
+
+$ python -m pip install -e .
+[...]
+Successfully installed annotated-types-0.7.0 docstring-parser-0.15 fractal-task-tools-0.0.1 pydantic-2.8.2 pydantic-core-2.20.1 typing-extensions-4.12.2
+
+$ fractal-manifest create --help
+usage: fractal-manifest create [-h] --package PACKAGE [--task-list-path TASK_LIST_PATH]
+
+Create new manifest file
+
+options:
+  -h, --help            show this help message and exit
+  --package PACKAGE     Example: 'fractal_tasks_core'
+  --task-list-path TASK_LIST_PATH
+                        Dot-separated path to the `task_list.py` module, relative to the package root (default value:
+                        'dev.task_list').
+
+```
+
+# Development
+
+```console
+$ python -m venv venv
+
+$ source venv/bin/activate
+
+$ python -m pip install -e .[dev]
+[...]
+Successfully installed asttokens-2.4.1 bumpver-2024.1130 click-8.1.8 colorama-0.4.6 coverage-7.6.12 devtools-0.12.2 exceptiongroup-1.2.2 executing-2.2.0 fractal-task-tools-0.0.1 iniconfig-2.0.0 lexid-2021.1006 packaging-24.2 pluggy-1.5.0 pygments-2.19.1 pytest-8.3.5 six-1.17.0 toml-0.10.2 tomli-2.2.1
+
+$ pre-commit install
+pre-commit installed at .git/hooks/pre-commit
+```
+
+## How to make a release
+From the development environment:
+```
+bumpver update --patch --dry
+```
+
+
+## Contributors and license
+
+Fractal was conceived in the Liberali Lab at the Friedrich Miescher Institute for Biomedical Research and in the Pelkmans Lab at the University of Zurich by [@jluethi](https://github.com/jluethi) and [@gusqgm](https://github.com/gusqgm). The Fractal project is now developed at the [BioVisionCenter](https://www.biovisioncenter.uzh.ch/en.html) at the University of Zurich and the project lead is with [@jluethi](https://github.com/jluethi). The core development is done under contract by [eXact lab S.r.l.](https://www.exact-lab.it).
+
+Unless otherwise specified, Fractal components are released under the BSD 3-Clause License, and copyright is with the BioVisionCenter at the University of Zurich.

fractal_task_tools-0.0.2/README.md

@@ -0,0 +1,65 @@
+# Fractal task tools
+
+[![PyPI version](https://img.shields.io/pypi/v/fractal-task-tools?color=gree)](https://pypi.org/project/fractal-task-tools/)
+[![License](https://img.shields.io/badge/License-BSD_3--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)
+[![CI Status](https://github.com/fractal-analytics-platform/fractal-task-tools/actions/workflows/ci.yml/badge.svg)](https://github.com/fractal-analytics-platform/fractal-task-tools/actions/workflows/ci_pip.yml)
+[![Coverage](https://raw.githubusercontent.com/fractal-analytics-platform/fractal-task-tools/python-coverage-comment-action-data/badge.svg)](https://htmlpreview.github.io/?https://github.com/fractal-analytics-platform/fractal-task-tools/blob/python-coverage-comment-action-data/htmlcov/index.html)
+
+Fractal-task-tools provides some basic tools for building tasks for the [Fractal](https://fractal-analytics-platform.github.io/) framework.
+
+![Fractal_overview_small](https://github.com/user-attachments/assets/666c8797-2594-4b8e-b1d2-b43fca66d1df)
+
+[Fractal](https://fractal-analytics-platform.github.io/) is a framework developed at the [BioVisionCenter](https://www.biovisioncenter.uzh.ch/en.html) to process bioimaging data at scale in the OME-Zarr format and prepare the images for interactive visualization.
+
+
+# Get started
+```console
+$ python -m venv venv
+
+$ source venv/bin/activate
+
+$ python -m pip install -e .
+[...]
+Successfully installed annotated-types-0.7.0 docstring-parser-0.15 fractal-task-tools-0.0.1 pydantic-2.8.2 pydantic-core-2.20.1 typing-extensions-4.12.2
+
+$ fractal-manifest create --help
+usage: fractal-manifest create [-h] --package PACKAGE [--task-list-path TASK_LIST_PATH]
+
+Create new manifest file
+
+options:
+  -h, --help            show this help message and exit
+  --package PACKAGE     Example: 'fractal_tasks_core'
+  --task-list-path TASK_LIST_PATH
+                        Dot-separated path to the `task_list.py` module, relative to the package root (default value:
+                        'dev.task_list').
+
+```
+
+# Development
+
+```console
+$ python -m venv venv
+
+$ source venv/bin/activate
+
+$ python -m pip install -e .[dev]
+[...]
+Successfully installed asttokens-2.4.1 bumpver-2024.1130 click-8.1.8 colorama-0.4.6 coverage-7.6.12 devtools-0.12.2 exceptiongroup-1.2.2 executing-2.2.0 fractal-task-tools-0.0.1 iniconfig-2.0.0 lexid-2021.1006 packaging-24.2 pluggy-1.5.0 pygments-2.19.1 pytest-8.3.5 six-1.17.0 toml-0.10.2 tomli-2.2.1
+
+$ pre-commit install
+pre-commit installed at .git/hooks/pre-commit
+```
+
+## How to make a release
+From the development environment:
+```
+bumpver update --patch --dry
+```
+
+
+## Contributors and license
+
+Fractal was conceived in the Liberali Lab at the Friedrich Miescher Institute for Biomedical Research and in the Pelkmans Lab at the University of Zurich by [@jluethi](https://github.com/jluethi) and [@gusqgm](https://github.com/gusqgm). The Fractal project is now developed at the [BioVisionCenter](https://www.biovisioncenter.uzh.ch/en.html) at the University of Zurich and the project lead is with [@jluethi](https://github.com/jluethi). The core development is done under contract by [eXact lab S.r.l.](https://www.exact-lab.it).
+
+Unless otherwise specified, Fractal components are released under the BSD 3-Clause License, and copyright is with the BioVisionCenter at the University of Zurich.

fractal_task_tools-0.0.2/pyproject.toml

@@ -0,0 +1,57 @@
+[project]
+name = "fractal-task-tools"
+version = "0.0.2"
+description = "Shared tools for Fractal tasks"
+authors = [
+    {name="Tommaso Comparin", email="tommaso.comparin@exact-lab.it"},
+]
+readme = "README.md"
+license = {text ="BSD-3-Clause"}
+requires-python = ">=3.10,<3.13"
+dependencies = [
+    "pydantic >=2.0.0,<=2.8.2",
+    "docstring-parser ==0.15",
+]
+[project.optional-dependencies]
+dev = [
+    "bumpver ==2024.1130",
+    "devtools ==0.12.2",
+    "pytest >=8.3.0, <9.0.0",
+    "coverage >=7.6.0,<7.7.0",
+]
+
+[project.urls]
+homepage = "https://github.com/fractal-analytics-platform/fractal-task-tools"
+repository = "https://github.com/fractal-analytics-platform/fractal-task-tools"
+changelog = "https://github.com/fractal-analytics-platform/fractal-task-tools/blob/main/CHANGELOG.md"
+
+
+[build-system]
+requires = ["setuptools >= 61.0"]
+build-backend = "setuptools.build_meta"
+
+[project.scripts]
+fractal-manifest = "fractal_task_tools._cli:main"
+
+[tool.bumpver]
+current_version = "0.0.2"
+version_pattern = "MAJOR.MINOR.PATCH[PYTAGNUM]"
+commit_message = "bump version {old_version} -> {new_version}"
+commit = true
+tag = true
+push = true
+
+[tool.bumpver.file_patterns]
+"pyproject.toml" = ['version = "{version}"$']
+"src/fractal_task_tools/__init__.py" = ['__VERSION__ = "{version}"$']
+
+
+[tool.coverage.run]
+branch = true
+parallel = true
+relative_files = true
+source = ["src"]
+omit = ["tests/*", "*/.venv/*"]
+
+[tool.coverage.report]
+omit = ["tests/*", "*/.venv/*"]

fractal_task_tools-0.0.2/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

fractal_task_tools-0.0.2/src/fractal_task_tools/__init__.py

@@ -0,0 +1,5 @@
+import logging
+
+logging.basicConfig(level=logging.INFO)
+
+__VERSION__ = "0.0.2"

fractal_task_tools-0.0.2/src/fractal_task_tools/_args_schemas.py

@@ -0,0 +1,214 @@
+import logging
+import os
+from collections import Counter
+from pathlib import Path
+from typing import Any
+from typing import Callable
+from typing import Optional
+
+from docstring_parser import parse as docparse
+from pydantic._internal import _generate_schema
+from pydantic._internal import _typing_extra
+from pydantic._internal._config import ConfigWrapper
+
+from ._descriptions import _get_class_attrs_descriptions
+from ._descriptions import _get_function_args_descriptions
+from ._descriptions import _insert_class_attrs_descriptions
+from ._descriptions import _insert_function_args_descriptions
+from ._pydantic_generatejsonschema import CustomGenerateJsonSchema
+from ._signature_constraints import _extract_function
+from ._signature_constraints import _validate_function_signature
+from ._titles import _include_titles
+
+
+_Schema = dict[str, Any]
+
+
+def _remove_attributes_from_descriptions(old_schema: _Schema) -> _Schema:
+    """
+    Keeps only the description part of the docstrings: e.g from
+    ```
+    'Custom class for Omero-channel window, based on OME-NGFF v0.4.\\n'
+    '\\n'
+    'Attributes:\\n'
+    'min: Do not change. It will be set to `0` by default.\\n'
+    'max: Do not change. It will be set according to bitdepth of the images\\n'
+    '    by default (e.g. 65535 for 16 bit images).\\n'
+    'start: Lower-bound rescaling value for visualization.\\n'
+    'end: Upper-bound rescaling value for visualization.'
+    ```
+    to `'Custom class for Omero-channel window, based on OME-NGFF v0.4.\\n'`.
+
+    Args:
+        old_schema: TBD
+    """
+    new_schema = old_schema.copy()
+    if "$defs" in new_schema:
+        for name, definition in new_schema["$defs"].items():
+            if "description" in definition.keys():
+                parsed_docstring = docparse(definition["description"])
+                new_schema["$defs"][name][
+                    "description"
+                ] = parsed_docstring.short_description
+            elif "title" in definition.keys():
+                title = definition["title"]
+                new_schema["$defs"][name][
+                    "description"
+                ] = f"Missing description for {title}."
+            else:
+                new_schema["$defs"][name][
+                    "description"
+                ] = "Missing description"
+    logging.info("[_remove_attributes_from_descriptions] END")
+    return new_schema
+
+
+def _create_schema_for_function(function: Callable) -> _Schema:
+    namespace = _typing_extra.add_module_globals(function, None)
+    gen_core_schema = _generate_schema.GenerateSchema(
+        ConfigWrapper(None), namespace
+    )
+    core_schema = gen_core_schema.generate_schema(function)
+    clean_core_schema = gen_core_schema.clean_schema(core_schema)
+    gen_json_schema = CustomGenerateJsonSchema()
+    json_schema = gen_json_schema.generate(
+        clean_core_schema, mode="validation"
+    )
+    return json_schema
+
+
+def create_schema_for_single_task(
+    executable: str,
+    package: Optional[str] = None,
+    pydantic_models: Optional[list[tuple[str, str, str]]] = None,
+    task_function: Optional[Callable] = None,
+    verbose: bool = False,
+) -> _Schema:
+    """
+    Main function to create a JSON Schema of task arguments
+
+    This function can be used in two ways:
+
+    1. `task_function` argument is `None`, `package` is set, and `executable`
+        is a path relative to that package.
+    2. `task_function` argument is provided, `executable` is an absolute path
+        to the function module, and `package` is `None. This is useful for
+        testing.
+    """
+
+    DEFINITIONS_KEY = "$defs"
+
+    logging.info("[create_schema_for_single_task] START")
+    if task_function is None:
+        usage = "1"
+        # Usage 1 (standard)
+        if package is None:
+            raise ValueError(
+                "Cannot call `create_schema_for_single_task with "
+                f"{task_function=} and {package=}. Exit."
+            )
+        if os.path.isabs(executable):
+            raise ValueError(
+                "Cannot call `create_schema_for_single_task with "
+                f"{task_function=} and absolute {executable=}. Exit."
+            )
+    else:
+        usage = "2"
+        # Usage 2 (testing)
+        if package is not None:
+            raise ValueError(
+                "Cannot call `create_schema_for_single_task with "
+                f"{task_function=} and non-None {package=}. Exit."
+            )
+        if not os.path.isabs(executable):
+            raise ValueError(
+                "Cannot call `create_schema_for_single_task with "
+                f"{task_function=} and non-absolute {executable=}. Exit."
+            )
+
+    # Extract function from module
+    if usage == "1":
+        # Extract the function name (for the moment we assume the function has
+        # the same name as the module)
+        function_name = Path(executable).with_suffix("").name
+        # Extract the function object
+        task_function = _extract_function(
+            package_name=package,
+            module_relative_path=executable,
+            function_name=function_name,
+            verbose=verbose,
+        )
+    else:
+        # The function object is already available, extract its name
+        function_name = task_function.__name__
+
+    if verbose:
+        logging.info(f"[create_schema_for_single_task] {function_name=}")
+        logging.info(f"[create_schema_for_single_task] {task_function=}")
+
+    # Validate function signature against some custom constraints
+    _validate_function_signature(task_function)
+
+    # Create and clean up schema
+    schema = _create_schema_for_function(task_function)
+    schema = _remove_attributes_from_descriptions(schema)
+
+    # Include titles for custom-model-typed arguments
+    schema = _include_titles(
+        schema, definitions_key=DEFINITIONS_KEY, verbose=verbose
+    )
+
+    # Include main title
+    if schema.get("title") is None:
+
+        def to_camel_case(snake_str):
+            return "".join(
+                x.capitalize() for x in snake_str.lower().split("_")
+            )
+
+        schema["title"] = to_camel_case(task_function.__name__)
+
+    # Include descriptions of function. Note: this function works both
+    # for usages 1 or 2 (see docstring).
+    function_args_descriptions = _get_function_args_descriptions(
+        package_name=package,
+        module_path=executable,
+        function_name=function_name,
+        verbose=verbose,
+    )
+
+    schema = _insert_function_args_descriptions(
+        schema=schema, descriptions=function_args_descriptions
+    )
+
+    if pydantic_models is not None:
+        # Check that model names are unique
+        pydantic_models_names = [item[2] for item in pydantic_models]
+        duplicate_class_names = [
+            name
+            for name, count in Counter(pydantic_models_names).items()
+            if count > 1
+        ]
+        if duplicate_class_names:
+            pydantic_models_str = "  " + "\n  ".join(map(str, pydantic_models))
+            raise ValueError(
+                "Cannot parse docstrings for models with non-unique names "
+                f"{duplicate_class_names}, in\n{pydantic_models_str}"
+            )
+
+        # Extract model-attribute descriptions and insert them into schema
+        for package_name, module_relative_path, class_name in pydantic_models:
+            attrs_descriptions = _get_class_attrs_descriptions(
+                package_name=package_name,
+                module_relative_path=module_relative_path,
+                class_name=class_name,
+            )
+            schema = _insert_class_attrs_descriptions(
+                schema=schema,
+                class_name=class_name,
+                descriptions=attrs_descriptions,
+                definition_key=DEFINITIONS_KEY,
+            )
+
+    logging.info("[create_schema_for_single_task] END")
+    return schema

fractal_task_tools-0.0.2/src/fractal_task_tools/_cli.py

@@ -0,0 +1,72 @@
+import argparse as ap
+import sys
+
+from fractal_task_tools._create_manifest import check_manifest_unchanged
+from fractal_task_tools._create_manifest import create_manifest
+from fractal_task_tools._create_manifest import write_manifest_to_file
+
+
+main_parser = ap.ArgumentParser(
+    description="`fractal-manifest` command-line interface",
+    allow_abbrev=False,
+)
+
+subparsers = main_parser.add_subparsers(
+    title="Available commands",
+    dest="cmd",
+)
+
+
+create_manifest_parser = subparsers.add_parser(
+    "create",
+    description="Create new manifest file",
+    allow_abbrev=False,
+)
+
+check_manifest_parser = subparsers.add_parser(
+    "check",
+    description="Check existing manifest file",
+    allow_abbrev=False,
+)
+
+
+for subparser in (create_manifest_parser, check_manifest_parser):
+    subparser.add_argument(
+        "--package",
+        type=str,
+        help="Example: 'fractal_tasks_core'",
+        required=True,
+    )
+    subparser.add_argument(
+        "--task-list-path",
+        type=str,
+        help=(
+            "Dot-separated path to the `task_list.py` module, "
+            "relative to the package root (default value: 'dev.task_list')."
+        ),
+        default="dev.task_list",
+        required=False,
+    )
+
+
+def main():
+    args = main_parser.parse_args(sys.argv[1:])
+    if args.cmd == "create":
+        manifest = create_manifest(
+            raw_package_name=args.package,
+            task_list_path=args.task_list_path,
+        )
+        write_manifest_to_file(
+            raw_package_name=args.package,
+            manifest=manifest,
+        )
+
+    elif args.cmd == "check":
+        manifest = create_manifest(
+            raw_package_name=args.package,
+            task_list_path=args.task_list_path,
+        )
+        check_manifest_unchanged(
+            raw_package_name=args.package,
+            manifest=manifest,
+        )