ddeutil-workflow 0.0.83__tar.gz → 0.0.85__tar.gz

{ddeutil_workflow-0.0.83 → ddeutil_workflow-0.0.85}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ddeutil-workflow
-Version: 0.0.83
+Version: 0.0.85
 Summary: Lightweight workflow orchestration with YAML template
 Author-email: ddeutils <korawich.anu@gmail.com>
 License: MIT
@@ -303,21 +303,18 @@ it will use default value and do not raise any error to you.
 > The config value that you will set on the environment should combine with
 > prefix, component, and name which is `WORKFLOW_{component}_{name}` (Upper case).
 
-| Name                        | Component | Default                                                                                                                         | Description                                                                                      |
-|:----------------------------|:---------:|:--------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|
-| **REGISTRY_CALLER**         |   CORE    | `.`                                                                                                                             | List of importable string for the call stage.                                                    |
-| **REGISTRY_FILTER**         |   CORE    | `ddeutil.workflow.templates`                                                                                                    | List of importable string for the filter template.                                               |
-| **CONF_PATH**               |   CORE    | `./conf`                                                                                                                        | The config path that keep all template `.yaml` files.                                            |
-| **STAGE_DEFAULT_ID**        |   CORE    | `false`                                                                                                                         | A flag that enable default stage ID that use for catch an execution output.                      |
-| **GENERATE_ID_SIMPLE_MODE** |   CORE    | `true`                                                                                                                          | A flog that enable generating ID with `md5` algorithm.                                           |
-| **DEBUG_MODE**              |    LOG    | `true`                                                                                                                          | A flag that enable logging with debug level mode.                                                |
-| **TIMEZONE**                |    LOG    | `Asia/Bangkok`                                                                                                                  | A Timezone string value that will pass to `ZoneInfo` object.                                     |
-| **FORMAT**                  |    LOG    | `%(asctime)s.%(msecs)03d (%(name)-10s, %(process)-5d,%(thread)-5d) [%(levelname)-7s] %(message)-120s (%(filename)s:%(lineno)s)` | A trace message console format.                                                                  |
-| **FORMAT_FILE**             |    LOG    | `{datetime} ({process:5d}, {thread:5d}) {message:120s} ({filename}:{lineno})`                                                   | A trace message format that use to write to target pointer.                                      |
-| **DATETIME_FORMAT**         |    LOG    | `%Y-%m-%d %H:%M:%S`                                                                                                             | A datetime format of the trace log.                                                              |
-| **TRACE_HANDLERS**          |    LOG    | `[{"type": "console"}]`                                                                                                         | A pointer URL of trace log that use to emit log message. Now uses optimized handler by default.  |
-| **AUDIT_CONF**              |    LOG    | `{"type": "file", "path": "./audits"}`                                                                                          | A pointer URL of audit log that use to write audit metrix.                                       |
-| **AUDIT_ENABLE_WRITE**      |    LOG    | `true`                                                                                                                          | A flag that enable writing audit log after end execution in the workflow release step.           |
+| Name                        | Component | Default                                | Description                                                                            |
+|:----------------------------|:---------:|:---------------------------------------|:---------------------------------------------------------------------------------------|
+| **REGISTRY_CALLER**         |   CORE    | `.`                                    | List of importable string for the call stage.                                          |
+| **REGISTRY_FILTER**         |   CORE    | `ddeutil.workflow.templates`           | List of importable string for the filter template.                                     |
+| **CONF_PATH**               |   CORE    | `./conf`                               | The config path that keep all template `.yaml` files.                                  |
+| **STAGE_DEFAULT_ID**        |   CORE    | `false`                                | A flag that enable default stage ID that use for catch an execution output.            |
+| **GENERATE_ID_SIMPLE_MODE** |   CORE    | `true`                                 | A flog that enable generating ID with `md5` algorithm.                                 |
+| **DEBUG_MODE**              |    LOG    | `true`                                 | A flag that enable logging with debug level mode.                                      |
+| **TIMEZONE**                |    LOG    | `Asia/Bangkok`                         | A Timezone string value that will pass to `ZoneInfo` object.                           |
+| **TRACE_HANDLERS**          |    LOG    | `[{"type": "console"}]`                | A Json string of list of trace handler config data that use to emit log message.       |
+| **AUDIT_CONF**              |    LOG    | `{"type": "file", "path": "./audits"}` | A Json string of audit config data that use to write audit metrix.                     |
+| **AUDIT_ENABLE_WRITE**      |    LOG    | `true`                                 | A flag that enable writing audit log after end execution in the workflow release step. |
 
 ## :rocket: Deployment
 

{ddeutil_workflow-0.0.83 → ddeutil_workflow-0.0.85}/README.md

@@ -253,21 +253,18 @@ it will use default value and do not raise any error to you.
 > The config value that you will set on the environment should combine with
 > prefix, component, and name which is `WORKFLOW_{component}_{name}` (Upper case).
 
-| Name                        | Component | Default                                                                                                                         | Description                                                                                      |
-|:----------------------------|:---------:|:--------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|
-| **REGISTRY_CALLER**         |   CORE    | `.`                                                                                                                             | List of importable string for the call stage.                                                    |
-| **REGISTRY_FILTER**         |   CORE    | `ddeutil.workflow.templates`                                                                                                    | List of importable string for the filter template.                                               |
-| **CONF_PATH**               |   CORE    | `./conf`                                                                                                                        | The config path that keep all template `.yaml` files.                                            |
-| **STAGE_DEFAULT_ID**        |   CORE    | `false`                                                                                                                         | A flag that enable default stage ID that use for catch an execution output.                      |
-| **GENERATE_ID_SIMPLE_MODE** |   CORE    | `true`                                                                                                                          | A flog that enable generating ID with `md5` algorithm.                                           |
-| **DEBUG_MODE**              |    LOG    | `true`                                                                                                                          | A flag that enable logging with debug level mode.                                                |
-| **TIMEZONE**                |    LOG    | `Asia/Bangkok`                                                                                                                  | A Timezone string value that will pass to `ZoneInfo` object.                                     |
-| **FORMAT**                  |    LOG    | `%(asctime)s.%(msecs)03d (%(name)-10s, %(process)-5d,%(thread)-5d) [%(levelname)-7s] %(message)-120s (%(filename)s:%(lineno)s)` | A trace message console format.                                                                  |
-| **FORMAT_FILE**             |    LOG    | `{datetime} ({process:5d}, {thread:5d}) {message:120s} ({filename}:{lineno})`                                                   | A trace message format that use to write to target pointer.                                      |
-| **DATETIME_FORMAT**         |    LOG    | `%Y-%m-%d %H:%M:%S`                                                                                                             | A datetime format of the trace log.                                                              |
-| **TRACE_HANDLERS**          |    LOG    | `[{"type": "console"}]`                                                                                                         | A pointer URL of trace log that use to emit log message. Now uses optimized handler by default.  |
-| **AUDIT_CONF**              |    LOG    | `{"type": "file", "path": "./audits"}`                                                                                          | A pointer URL of audit log that use to write audit metrix.                                       |
-| **AUDIT_ENABLE_WRITE**      |    LOG    | `true`                                                                                                                          | A flag that enable writing audit log after end execution in the workflow release step.           |
+| Name                        | Component | Default                                | Description                                                                            |
+|:----------------------------|:---------:|:---------------------------------------|:---------------------------------------------------------------------------------------|
+| **REGISTRY_CALLER**         |   CORE    | `.`                                    | List of importable string for the call stage.                                          |
+| **REGISTRY_FILTER**         |   CORE    | `ddeutil.workflow.templates`           | List of importable string for the filter template.                                     |
+| **CONF_PATH**               |   CORE    | `./conf`                               | The config path that keep all template `.yaml` files.                                  |
+| **STAGE_DEFAULT_ID**        |   CORE    | `false`                                | A flag that enable default stage ID that use for catch an execution output.            |
+| **GENERATE_ID_SIMPLE_MODE** |   CORE    | `true`                                 | A flog that enable generating ID with `md5` algorithm.                                 |
+| **DEBUG_MODE**              |    LOG    | `true`                                 | A flag that enable logging with debug level mode.                                      |
+| **TIMEZONE**                |    LOG    | `Asia/Bangkok`                         | A Timezone string value that will pass to `ZoneInfo` object.                           |
+| **TRACE_HANDLERS**          |    LOG    | `[{"type": "console"}]`                | A Json string of list of trace handler config data that use to emit log message.       |
+| **AUDIT_CONF**              |    LOG    | `{"type": "file", "path": "./audits"}` | A Json string of audit config data that use to write audit metrix.                     |
+| **AUDIT_ENABLE_WRITE**      |    LOG    | `true`                                 | A flag that enable writing audit log after end execution in the workflow release step. |
 
 ## :rocket: Deployment
 

{ddeutil_workflow-0.0.83 → ddeutil_workflow-0.0.85}/src/ddeutil/workflow/__about__.py

@@ -1,2 +1,2 @@
-__version__: str = "0.0.83"
+__version__: str = "0.0.85"
 __python_version__: str = "3.9"

{ddeutil_workflow-0.0.83 → ddeutil_workflow-0.0.85}/src/ddeutil/workflow/__init__.py

@@ -107,10 +107,10 @@ from .job import (
     Rule,
     RunsOnModel,
     Strategy,
-    docker_execution,
-    local_execute,
-    local_execute_strategy,
-    self_hosted_execute,
+    docker_process,
+    local_process,
+    local_process_strategy,
+    self_hosted_process,
 )
 from .params import (
     ArrayParam,

{ddeutil_workflow-0.0.83 → ddeutil_workflow-0.0.85}/src/ddeutil/workflow/audits.py

@@ -60,7 +60,7 @@ from typing_extensions import Self
 
 from .__types import DictData
 from .conf import dynamic
-from .traces import Trace, get_trace, set_logging
+from .traces import Trace, get_trace
 
 logger = logging.getLogger("ddeutil.workflow")
 
@@ -129,7 +129,6 @@ class BaseAudit(BaseModel, ABC):
     """
 
     type: Literal["base"] = "base"
-    logging_name: str = "ddeutil.workflow"
     extras: DictData = Field(
         default_factory=dict,
         description="An extras parameter that want to override core config",
@@ -152,9 +151,6 @@ class BaseAudit(BaseModel, ABC):
         """
         if dynamic("enable_write_audit", extras=self.extras):
             self.do_before()
-
-        # NOTE: Start setting log config in this line with cache.
-        set_logging(self.logging_name)
         return self
 
     @abstractmethod
@@ -795,7 +791,13 @@ Audit = Annotated[
         LocalFileAudit,
         LocalSQLiteAudit,
     ],
-    Field(discriminator="type"),
+    Field(
+        discriminator="type",
+        description=(
+            "An union of supported Audit model that have inherited from "
+            "BaseAudit."
+        ),
+    ),
 ]
 
 

{ddeutil_workflow-0.0.83 → ddeutil_workflow-0.0.85}/src/ddeutil/workflow/conf.py

@@ -101,7 +101,8 @@ class Config:  # pragma: no cov
         """Register Filter that is a list of importable string for the filter
         template.
 
-        :rtype: list[str]
+        Returns:
+            list[str]: A list of module import string.
         """
         regis_filter_str: str = env(
             "CORE_REGISTRY_FILTER", "ddeutil.workflow.templates"
@@ -130,17 +131,6 @@ class Config:  # pragma: no cov
         """
         return ZoneInfo(env("LOG_TIMEZONE", "UTC"))
 
-    @property
-    def log_format(self) -> str:
-        return env(
-            "LOG_FORMAT",
-            (
-                "%(asctime)s.%(msecs)03d (%(process)-5d, "
-                "%(thread)-5d) [%(levelname)-7s] (%(cut_id)s) %(message)-120s "
-                "(%(filename)s:%(lineno)s) (%(name)-10s)"
-            ),
-        )
-
     @property
     def audit_conf(self) -> dict[str, Any]:
         return json.loads(
@@ -151,10 +141,6 @@ class Config:  # pragma: no cov
     def enable_write_audit(self) -> bool:
         return str2bool(env("LOG_AUDIT_ENABLE_WRITE", "false"))
 
-    @property
-    def log_datetime_format(self) -> str:
-        return env("LOG_DATETIME_FORMAT", "%Y-%m-%d %H:%M:%S")
-
     @property
     def stage_default_id(self) -> bool:
         return str2bool(env("CORE_STAGE_DEFAULT_ID", "false"))
@@ -441,7 +427,8 @@ class YamlParser:
         """Return object of string type which implement on any registry. The
         object type.
 
-        :rtype: str
+        Returns:
+            str: A type that get from config data.
         """
         if _typ := self.data.get("type"):
             return _typ

{ddeutil_workflow-0.0.83 → ddeutil_workflow-0.0.85}/src/ddeutil/workflow/errors.py

@@ -8,17 +8,6 @@
 This module provides a comprehensive exception hierarchy for the workflow system.
 The exceptions are designed to be lightweight while providing sufficient context
 for error handling and debugging.
-
-Classes:
-    BaseError: Base exception class with context support
-    StageError: Exceptions related to stage execution
-    JobError: Exceptions related to job execution
-    WorkflowError: Exceptions related to workflow execution
-    ParamError: Exceptions related to parameter validation
-    ResultError: Exceptions related to result processing
-
-Functions:
-    to_dict: Convert exception instances to dictionary format
 """
 from __future__ import annotations
 
@@ -58,11 +47,13 @@ def to_dict(exception: Exception, **kwargs) -> ErrorData:  # pragma: no cov
     Example:
         >>> try:
         >>>     raise ValueError("Something went wrong")
-        >>> except Exception as e:
-        >>>     error_data = to_dict(e, context="workflow_execution")
-        >>>     # Returns: {
-        >>>     #   "name": "ValueError", "message": "Something went wrong", "context": "workflow_execution"
-        >>>     # }
+        >>> except Exception as err:
+        >>>     error_data = to_dict(err, context="workflow_execution")
+        {
+            "name": "ValueError",
+            "message": "Something went wrong",
+            "context": "workflow_execution",
+        }
     """
     return {
         "name": exception.__class__.__name__,
@@ -71,6 +62,20 @@ def to_dict(exception: Exception, **kwargs) -> ErrorData:  # pragma: no cov
     }
 
 
+def mark_errors(context: DictData, error: JobError) -> None:
+    """Make the errors context result with the refs value depends on the nested
+    execute func.
+
+    Args:
+        context (DictData): A context data.
+        error (JobError): A stage exception object.
+    """
+    if "errors" in context:
+        context["errors"][error.refs] = error.to_dict()
+    else:
+        context["errors"] = error.to_dict(with_refs=True)
+
+
 class BaseError(Exception):
     """Base exception class for all workflow-related errors.
 
@@ -80,9 +85,14 @@ class BaseError(Exception):
     for debugging purposes.
 
     Attributes:
-        refs: Optional reference identifier for error correlation
-        context: Additional context data related to the error
-        params: Parameter data that was being processed when error occurred
+        refs (str | int, default None): Optional reference identifier for error
+            correlation
+        context (DictData, default None): Additional context data related to the
+            error
+        params (DictData, default None): Parameter data that was being processed
+            when error occurred
+        allow_traceback (bool, default True): A flag for printing traceback
+            after it catch this object.
 
     Example:
         >>> try:
@@ -100,11 +110,13 @@ class BaseError(Exception):
         refs: Optional[StrOrInt] = None,
         context: Optional[DictData] = None,
         params: Optional[DictData] = None,
+        allow_traceback: bool = True,
     ) -> None:
         super().__init__(message)
         self.refs: Optional[str] = refs
         self.context: DictData = context or {}
         self.params: DictData = params or {}
+        self.allow_traceback: bool = allow_traceback
 
     @overload
     def to_dict(