ddeutil-workflow 0.0.8__tar.gz → 0.0.9__tar.gz

ddeutil_workflow-0.0.9/PKG-INFO

@@ -0,0 +1,273 @@
+Metadata-Version: 2.1
+Name: ddeutil-workflow
+Version: 0.0.9
+Summary: Lightweight workflow orchestration with less dependencies
+Author-email: ddeutils <korawich.anu@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/ddeutils/ddeutil-workflow/
+Project-URL: Source Code, https://github.com/ddeutils/ddeutil-workflow/
+Keywords: orchestration,workflow
+Classifier: Topic :: Utilities
+Classifier: Natural Language :: English
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.9.13
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: fmtutil
+Requires-Dist: ddeutil-io
+Requires-Dist: python-dotenv==1.0.1
+Requires-Dist: typer==0.12.4
+Provides-Extra: schedule
+Requires-Dist: schedule<2.0.0,==1.2.2; extra == "schedule"
+Provides-Extra: api
+Requires-Dist: fastapi[standard]==0.112.1; extra == "api"
+Requires-Dist: croniter==3.0.3; extra == "api"
+
+# Workflow
+
+[![test](https://github.com/ddeutils/ddeutil-workflow/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/ddeutils/ddeutil-workflow/actions/workflows/tests.yml)
+[![python support version](https://img.shields.io/pypi/pyversions/ddeutil-workflow)](https://pypi.org/project/ddeutil-workflow/)
+[![size](https://img.shields.io/github/languages/code-size/ddeutils/ddeutil-workflow)](https://github.com/ddeutils/ddeutil-workflow)
+[![gh license](https://img.shields.io/github/license/ddeutils/ddeutil-workflow)](https://github.com/ddeutils/ddeutil-workflow/blob/main/LICENSE)
+[![code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+
+**Table of Contents**:
+
+- [Installation](#installation)
+- [Getting Started](#getting-started)
+  - [On](#on)
+  - [Pipeline](#pipeline)
+- [Usage](#usage)
+- [Configuration](#configuration)
+- [Future](#future)
+- [Deployment](#deployment)
+
+The **Lightweight workflow orchestration** with less dependencies the was created
+for easy to make a simple metadata driven for data pipeline orchestration.
+It can to use for data operator by a `.yaml` template.
+
+> [!WARNING]
+> This package provide only orchestration workload. That mean you should not use
+> workflow stage to process any large data which use lot of compute usecase.
+
+In my opinion, I think it should not create duplicate pipeline codes if I can
+write with dynamic input parameters on the one template pipeline that just change
+the input parameters per use-case instead.
+This way I can handle a lot of logical pipelines in our orgs with only metadata
+configuration. It called **Metadata Driven Data Pipeline**.
+
+Next, we should get some monitoring tools for manage logging that return from
+pipeline running. Because it not show us what is a use-case that running data
+pipeline.
+
+> [!NOTE]
+> _Disclaimer_: I inspire the dynamic statement from the GitHub Action `.yml` files
+> and all of config file from several data orchestration framework tools from my
+> experience on Data Engineer.
+
+**Rules of This Workflow engine**:
+
+1. Minimum unit of scheduling is 1 minute
+2. Cannot re-run only failed stage and its pending downstream
+3. All parallel tasks inside workflow engine use Threading
+   (Because Python 3.13 unlock GIL)
+
+## Installation
+
+This project need `ddeutil-io` extension namespace packages. If you want to install
+this package with application add-ons, you should add `app` in installation;
+
+| Usecase           | Install Optional                         | Support            |
+|-------------------|------------------------------------------|--------------------|
+| Python & CLI      | `pip install ddeutil-workflow`           | :heavy_check_mark: |
+| Scheduler Service | `pip install ddeutil-workflow[schedule]` | :x:                |
+| FastAPI Server    | `pip install ddeutil-workflow[api]`      | :x:                |
+
+
+> I added this feature to the main milestone.
+>
+> **Docker Images** supported:
+>
+> | Docker Image                | Python Version | Support |
+> |-----------------------------|----------------|---------|
+> | ddeutil-workflow:latest     | `3.9`          | :x:     |
+> | ddeutil-workflow:python3.10 | `3.10`         | :x:     |
+> | ddeutil-workflow:python3.11 | `3.11`         | :x:     |
+> | ddeutil-workflow:python3.12 | `3.12`         | :x:     |
+
+## Getting Started
+
+The main feature of this project is the `Pipeline` object that can call any
+registries function. The pipeline can handle everything that you want to do, it
+will passing parameters and catching the output for re-use it to next step.
+
+### On
+
+The **On** is schedule object that receive crontab value and able to generate
+datetime value with next or previous with any start point of an input datetime.
+
+```yaml
+# This file should keep under this path: `./root-path/conf-path/*`
+on_every_5_min:
+  type: on.On
+  cron: "*/5 * * * *"
+```
+
+```python
+from ddeutil.workflow.on import On
+
+# NOTE: Start load the on data from `.yaml` template file with this key.
+schedule = On.from_loader(name='on_every_5_min', externals={})
+
+assert '*/5 * * * *' == str(schedule.cronjob)
+
+cron_iter = schedule.generate('2022-01-01 00:00:00')
+
+assert "2022-01-01 00:05:00" f"{cron_iter.next:%Y-%m-%d %H:%M:%S}"
+assert "2022-01-01 00:10:00" f"{cron_iter.next:%Y-%m-%d %H:%M:%S}"
+assert "2022-01-01 00:15:00" f"{cron_iter.next:%Y-%m-%d %H:%M:%S}"
+```
+
+### Pipeline
+
+The **Pipeline** object that is the core feature of this project.
+
+```yaml
+# This file should keep under this path: `./root-path/conf-path/*`
+pipeline-name:
+  type: ddeutil.workflow.pipeline.Pipeline
+  on: 'on_every_5_min'
+  params:
+    author-run:
+      type: str
+    run-date:
+      type: datetime
+  jobs:
+    first-job:
+      stages:
+        - name: "Empty stage do logging to console only!!"
+```
+
+```python
+from ddeutil.workflow.pipeline import Pipeline
+
+pipe = Pipeline.from_loader(name='pipeline-name', externals={})
+pipe.execute(params={'author-run': 'Local Workflow', 'run-date': '2024-01-01'})
+```
+
+> [!NOTE]
+> The above parameter can use short declarative statement. You can pass a parameter
+> type to the key of a parameter name but it does not handler default value if you
+> run this pipeline workflow with schedule.
+>
+> ```yaml
+> ...
+> params:
+>   author-run: str
+>   run-date: datetime
+> ...
+> ```
+>
+> And for the type, you can remove `ddeutil.workflow` prefix because we can find
+> it by looping search from `WORKFLOW_CORE_REGISTRY` value.
+
+## Usage
+
+This is examples that use workflow file for running common Data Engineering
+use-case.
+
+> [!IMPORTANT]
+> I recommend you to use the `hook` stage for all actions that you want to do
+> with pipeline activity that you want to orchestrate. Because it able to dynamic
+> an input argument with the same hook function that make you use less time to
+> maintenance your data pipelines.
+
+```yaml
+run_py_local:
+  type: pipeline.Pipeline
+  on:
+    - cronjob: '*/5 * * * *'
+      timezone: "Asia/Bangkok"
+  params:
+    author-run: str
+    run-date: datetime
+  jobs:
+    getting-api-data:
+      stages:
+        - name: "Retrieve API Data"
+          id: retrieve-api
+          uses: tasks/get-api-with-oauth-to-s3@requests
+          with:
+            url: https://open-data/
+            auth: ${API_ACCESS_REFRESH_TOKEN}
+            aws_s3_path: my-data/open-data/
+            # This Authentication code should implement with your custom hook function
+            aws_access_client_id: ${AWS_ACCESS_CLIENT_ID}
+            aws_access_client_secret: ${AWS_ACCESS_CLIENT_SECRET}
+```
+
+## Configuration
+
+| Environment                         | Component | Default                      | Description                                                                |
+|-------------------------------------|-----------|------------------------------|----------------------------------------------------------------------------|
+| `WORKFLOW_ROOT_PATH`                | Core      | .                            | The root path of the workflow application                                  |
+| `WORKFLOW_CORE_REGISTRY`            | Core      | ddeutil.workflow,tests.utils | List of importable string for the hook stage                               |
+| `WORKFLOW_CORE_REGISTRY_FILTER`     | Core      | ddeutil.workflow.utils       | List of importable string for the filter template                          |
+| `WORKFLOW_CORE_PATH_CONF`           | Core      | conf                         | The config path that keep all template `.yaml` files                       |
+| `WORKFLOW_CORE_TIMEZONE`            | Core      | Asia/Bangkok                 | A Timezone string value that will pass to `ZoneInfo` object                |
+| `WORKFLOW_CORE_STAGE_DEFAULT_ID`    | Core      | true                         | A flag that enable default stage ID that use for catch an execution output |
+| `WORKFLOW_CORE_STAGE_RAISE_ERROR`   | Core      | true                         | A flag that all stage raise StageException from stage execution            |
+| `WORKFLOW_CORE_MAX_PIPELINE_POKING` | Core      | 4                            |                                                                            |
+| `WORKFLOW_CORE_MAX_JOB_PARALLEL`    | Core      | 2                            | The maximum job number that able to run parallel in pipeline executor      |
+| `WORKFLOW_LOG_ENABLE_WRITE`         | Log       | true                         | A flag that enable logging object saving log to its destination            |
+
+
+**Application**:
+
+| Environment                         | Default | Description |
+|-------------------------------------|---------|-------------|
+| `WORKFLOW_APP_PROCESS_WORKER`       | 2       |             |
+| `WORKFLOW_APP_PIPELINE_PER_PROCESS` | 100     |             |
+
+**API server**:
+
+| Environment           | Default                                                | Description                                                        |
+|-----------------------|--------------------------------------------------------|--------------------------------------------------------------------|
+| `WORKFLOW_API_DB_URL` | postgresql+asyncpg://user:pass@localhost:5432/schedule | A Database URL that will pass to SQLAlchemy create_engine function |
+
+## Future
+
+The current milestone that will develop and necessary features that should to
+implement on this project.
+
+- ...
+
+## Deployment
+
+This package able to run as a application service for receive manual trigger
+from the master node via RestAPI or use to be Scheduler background service
+like crontab job but via Python API.
+
+### Schedule Service
+
+```shell
+(venv) $ python src.ddeutil.workflow.app
+```
+
+### API Server
+
+```shell
+(venv) $ uvicorn src.ddeutil.workflow.api:app --host 0.0.0.0 --port 80 --reload
+```
+
+> [!NOTE]
+> If this package already deploy, it able to use
+> `uvicorn ddeutil.workflow.api:app --host 0.0.0.0 --port 80`

ddeutil_workflow-0.0.9/README.md

@@ -0,0 +1,240 @@
+# Workflow
+
+[![test](https://github.com/ddeutils/ddeutil-workflow/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/ddeutils/ddeutil-workflow/actions/workflows/tests.yml)
+[![python support version](https://img.shields.io/pypi/pyversions/ddeutil-workflow)](https://pypi.org/project/ddeutil-workflow/)
+[![size](https://img.shields.io/github/languages/code-size/ddeutils/ddeutil-workflow)](https://github.com/ddeutils/ddeutil-workflow)
+[![gh license](https://img.shields.io/github/license/ddeutils/ddeutil-workflow)](https://github.com/ddeutils/ddeutil-workflow/blob/main/LICENSE)
+[![code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+
+**Table of Contents**:
+
+- [Installation](#installation)
+- [Getting Started](#getting-started)
+  - [On](#on)
+  - [Pipeline](#pipeline)
+- [Usage](#usage)
+- [Configuration](#configuration)
+- [Future](#future)
+- [Deployment](#deployment)
+
+The **Lightweight workflow orchestration** with less dependencies the was created
+for easy to make a simple metadata driven for data pipeline orchestration.
+It can to use for data operator by a `.yaml` template.
+
+> [!WARNING]
+> This package provide only orchestration workload. That mean you should not use
+> workflow stage to process any large data which use lot of compute usecase.
+
+In my opinion, I think it should not create duplicate pipeline codes if I can
+write with dynamic input parameters on the one template pipeline that just change
+the input parameters per use-case instead.
+This way I can handle a lot of logical pipelines in our orgs with only metadata
+configuration. It called **Metadata Driven Data Pipeline**.
+
+Next, we should get some monitoring tools for manage logging that return from
+pipeline running. Because it not show us what is a use-case that running data
+pipeline.
+
+> [!NOTE]
+> _Disclaimer_: I inspire the dynamic statement from the GitHub Action `.yml` files
+> and all of config file from several data orchestration framework tools from my
+> experience on Data Engineer.
+
+**Rules of This Workflow engine**:
+
+1. Minimum unit of scheduling is 1 minute
+2. Cannot re-run only failed stage and its pending downstream
+3. All parallel tasks inside workflow engine use Threading
+   (Because Python 3.13 unlock GIL)
+
+## Installation
+
+This project need `ddeutil-io` extension namespace packages. If you want to install
+this package with application add-ons, you should add `app` in installation;
+
+| Usecase           | Install Optional                         | Support            |
+|-------------------|------------------------------------------|--------------------|
+| Python & CLI      | `pip install ddeutil-workflow`           | :heavy_check_mark: |
+| Scheduler Service | `pip install ddeutil-workflow[schedule]` | :x:                |
+| FastAPI Server    | `pip install ddeutil-workflow[api]`      | :x:                |
+
+
+> I added this feature to the main milestone.
+>
+> **Docker Images** supported:
+>
+> | Docker Image                | Python Version | Support |
+> |-----------------------------|----------------|---------|
+> | ddeutil-workflow:latest     | `3.9`          | :x:     |
+> | ddeutil-workflow:python3.10 | `3.10`         | :x:     |
+> | ddeutil-workflow:python3.11 | `3.11`         | :x:     |
+> | ddeutil-workflow:python3.12 | `3.12`         | :x:     |
+
+## Getting Started
+
+The main feature of this project is the `Pipeline` object that can call any
+registries function. The pipeline can handle everything that you want to do, it
+will passing parameters and catching the output for re-use it to next step.
+
+### On
+
+The **On** is schedule object that receive crontab value and able to generate
+datetime value with next or previous with any start point of an input datetime.
+
+```yaml
+# This file should keep under this path: `./root-path/conf-path/*`
+on_every_5_min:
+  type: on.On
+  cron: "*/5 * * * *"
+```
+
+```python
+from ddeutil.workflow.on import On
+
+# NOTE: Start load the on data from `.yaml` template file with this key.
+schedule = On.from_loader(name='on_every_5_min', externals={})
+
+assert '*/5 * * * *' == str(schedule.cronjob)
+
+cron_iter = schedule.generate('2022-01-01 00:00:00')
+
+assert "2022-01-01 00:05:00" f"{cron_iter.next:%Y-%m-%d %H:%M:%S}"
+assert "2022-01-01 00:10:00" f"{cron_iter.next:%Y-%m-%d %H:%M:%S}"
+assert "2022-01-01 00:15:00" f"{cron_iter.next:%Y-%m-%d %H:%M:%S}"
+```
+
+### Pipeline
+
+The **Pipeline** object that is the core feature of this project.
+
+```yaml
+# This file should keep under this path: `./root-path/conf-path/*`
+pipeline-name:
+  type: ddeutil.workflow.pipeline.Pipeline
+  on: 'on_every_5_min'
+  params:
+    author-run:
+      type: str
+    run-date:
+      type: datetime
+  jobs:
+    first-job:
+      stages:
+        - name: "Empty stage do logging to console only!!"
+```
+
+```python
+from ddeutil.workflow.pipeline import Pipeline
+
+pipe = Pipeline.from_loader(name='pipeline-name', externals={})
+pipe.execute(params={'author-run': 'Local Workflow', 'run-date': '2024-01-01'})
+```
+
+> [!NOTE]
+> The above parameter can use short declarative statement. You can pass a parameter
+> type to the key of a parameter name but it does not handler default value if you
+> run this pipeline workflow with schedule.
+>
+> ```yaml
+> ...
+> params:
+>   author-run: str
+>   run-date: datetime
+> ...
+> ```
+>
+> And for the type, you can remove `ddeutil.workflow` prefix because we can find
+> it by looping search from `WORKFLOW_CORE_REGISTRY` value.
+
+## Usage
+
+This is examples that use workflow file for running common Data Engineering
+use-case.
+
+> [!IMPORTANT]
+> I recommend you to use the `hook` stage for all actions that you want to do
+> with pipeline activity that you want to orchestrate. Because it able to dynamic
+> an input argument with the same hook function that make you use less time to
+> maintenance your data pipelines.
+
+```yaml
+run_py_local:
+  type: pipeline.Pipeline
+  on:
+    - cronjob: '*/5 * * * *'
+      timezone: "Asia/Bangkok"
+  params:
+    author-run: str
+    run-date: datetime
+  jobs:
+    getting-api-data:
+      stages:
+        - name: "Retrieve API Data"
+          id: retrieve-api
+          uses: tasks/get-api-with-oauth-to-s3@requests
+          with:
+            url: https://open-data/
+            auth: ${API_ACCESS_REFRESH_TOKEN}
+            aws_s3_path: my-data/open-data/
+            # This Authentication code should implement with your custom hook function
+            aws_access_client_id: ${AWS_ACCESS_CLIENT_ID}
+            aws_access_client_secret: ${AWS_ACCESS_CLIENT_SECRET}
+```
+
+## Configuration
+
+| Environment                         | Component | Default                      | Description                                                                |
+|-------------------------------------|-----------|------------------------------|----------------------------------------------------------------------------|
+| `WORKFLOW_ROOT_PATH`                | Core      | .                            | The root path of the workflow application                                  |
+| `WORKFLOW_CORE_REGISTRY`            | Core      | ddeutil.workflow,tests.utils | List of importable string for the hook stage                               |
+| `WORKFLOW_CORE_REGISTRY_FILTER`     | Core      | ddeutil.workflow.utils       | List of importable string for the filter template                          |
+| `WORKFLOW_CORE_PATH_CONF`           | Core      | conf                         | The config path that keep all template `.yaml` files                       |
+| `WORKFLOW_CORE_TIMEZONE`            | Core      | Asia/Bangkok                 | A Timezone string value that will pass to `ZoneInfo` object                |
+| `WORKFLOW_CORE_STAGE_DEFAULT_ID`    | Core      | true                         | A flag that enable default stage ID that use for catch an execution output |
+| `WORKFLOW_CORE_STAGE_RAISE_ERROR`   | Core      | true                         | A flag that all stage raise StageException from stage execution            |
+| `WORKFLOW_CORE_MAX_PIPELINE_POKING` | Core      | 4                            |                                                                            |
+| `WORKFLOW_CORE_MAX_JOB_PARALLEL`    | Core      | 2                            | The maximum job number that able to run parallel in pipeline executor      |
+| `WORKFLOW_LOG_ENABLE_WRITE`         | Log       | true                         | A flag that enable logging object saving log to its destination            |
+
+
+**Application**:
+
+| Environment                         | Default | Description |
+|-------------------------------------|---------|-------------|
+| `WORKFLOW_APP_PROCESS_WORKER`       | 2       |             |
+| `WORKFLOW_APP_PIPELINE_PER_PROCESS` | 100     |             |
+
+**API server**:
+
+| Environment           | Default                                                | Description                                                        |
+|-----------------------|--------------------------------------------------------|--------------------------------------------------------------------|
+| `WORKFLOW_API_DB_URL` | postgresql+asyncpg://user:pass@localhost:5432/schedule | A Database URL that will pass to SQLAlchemy create_engine function |
+
+## Future
+
+The current milestone that will develop and necessary features that should to
+implement on this project.
+
+- ...
+
+## Deployment
+
+This package able to run as a application service for receive manual trigger
+from the master node via RestAPI or use to be Scheduler background service
+like crontab job but via Python API.
+
+### Schedule Service
+
+```shell
+(venv) $ python src.ddeutil.workflow.app
+```
+
+### API Server
+
+```shell
+(venv) $ uvicorn src.ddeutil.workflow.api:app --host 0.0.0.0 --port 80 --reload
+```
+
+> [!NOTE]
+> If this package already deploy, it able to use
+> `uvicorn ddeutil.workflow.api:app --host 0.0.0.0 --port 80`

{ddeutil_workflow-0.0.8 → ddeutil_workflow-0.0.9}/pyproject.toml

@@ -4,11 +4,11 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "ddeutil-workflow"
-description = "Data Developer & Engineer Workflow Utility Objects"
+description = "Lightweight workflow orchestration with less dependencies"
 readme = {file = "README.md", content-type = "text/markdown"}
 license = {text = "MIT"}
 authors = [{ name = "ddeutils", email = "korawich.anu@gmail.com" }]
-keywords = ['data', 'workflow', 'utility', 'pipeline']
+keywords = ['orchestration', 'workflow']
 classifiers = [
     "Topic :: Utilities",
     "Natural Language :: English",
@@ -28,16 +28,17 @@ dependencies = [
     "fmtutil",
     "ddeutil-io",
     "python-dotenv==1.0.1",
+    "typer==0.12.4",
 ]
 dynamic = ["version"]
 
 [project.optional-dependencies]
-app = [
+schedule = [
     "schedule==1.2.2,<2.0.0",
 ]
 api = [
-    "fastapi[standard]==0.112.0",
-    "apscheduler[sqlalchemy]==3.10.4,<4.0.0",
+    "fastapi[standard]==0.112.1",
+    # TODO: This package can migrate to use /cron/
     "croniter==3.0.3",
 ]
 
@@ -45,6 +46,9 @@ api = [
 Homepage = "https://github.com/ddeutils/ddeutil-workflow/"
 "Source Code" = "https://github.com/ddeutils/ddeutil-workflow/"
 
+[project.scripts]
+workflow = "ddeutil.workflow.cli:app"
+
 [tool.setuptools.dynamic]
 version = {attr = "ddeutil.workflow.__about__.__version__"}
 
@@ -62,8 +66,7 @@ concurrency = ["thread", "multiprocessing"]
 source = ["ddeutil.workflow", "tests"]
 omit = [
     "scripts/",
-    "src/ddeutil/workflow/api.py",
-    "src/ddeutil/workflow/app.py",
+    # Omit this files because it does not ready to production.
     "src/ddeutil/workflow/repeat.py",
     "src/ddeutil/workflow/route.py",
     "tests/utils.py",

ddeutil_workflow-0.0.9/src/ddeutil/workflow/__about__.py

@@ -0,0 +1 @@
+__version__: str = "0.0.9"

{ddeutil_workflow-0.0.8 → ddeutil_workflow-0.0.9}/src/ddeutil/workflow/__init__.py

@@ -10,22 +10,11 @@ from .exceptions import (
     StageException,
     UtilException,
 )
-from .on import AwsOn, On
-from .pipeline import Job, Pipeline
-from .stage import (
-    BashStage,
-    EmptyStage,
-    HookStage,
-    PyStage,
-    Stage,
-    TriggerStage,
-)
+from .on import On, interval2crontab
+from .pipeline import Job, Pipeline, Strategy
+from .stage import Stage, handler_result
 from .utils import (
-    ChoiceParam,
-    DatetimeParam,
-    IntParam,
     Param,
-    StrParam,
     dash2underscore,
     param2template,
 )

ddeutil_workflow-0.0.9/src/ddeutil/workflow/api.py

@@ -0,0 +1,89 @@
+# ------------------------------------------------------------------------------
+# Copyright (c) 2022 Korawich Anuttra. All rights reserved.
+# Licensed under the MIT License. See LICENSE in the project root for
+# license information.
+# ------------------------------------------------------------------------------
+from __future__ import annotations
+
+import asyncio
+import logging
+import os
+import uuid
+from queue import Empty, Queue
+
+from ddeutil.core import str2bool
+from dotenv import load_dotenv
+from fastapi import FastAPI
+from fastapi.middleware.gzip import GZipMiddleware
+from fastapi.responses import UJSONResponse
+from pydantic import BaseModel
+
+from .repeat import repeat_every
+
+load_dotenv()
+logger = logging.getLogger(__name__)
+logging.basicConfig(
+    level=logging.DEBUG,
+    format=(
+        "%(asctime)s.%(msecs)03d (%(name)-10s, %(process)-5d, %(thread)-5d) "
+        "[%(levelname)-7s] %(message)-120s (%(filename)s:%(lineno)s)"
+    ),
+    handlers=[logging.StreamHandler()],
+    datefmt="%Y-%m-%d %H:%M:%S",
+)
+
+
+app = FastAPI()
+app.add_middleware(GZipMiddleware, minimum_size=1000)
+app.queue = Queue()
+app.output_dict = {}
+app.queue_limit = 2
+
+
+@app.on_event("startup")
+@repeat_every(seconds=10, logger=logger)
+def broker_upper_messages():
+    """Broker for receive message from the `/upper` path and change it to upper
+    case.
+    """
+    for _ in range(app.queue_limit):
+        try:
+            obj = app.queue.get_nowait()
+            app.output_dict[obj["request_id"]] = obj["text"].upper()
+            logger.info(f"Upper message: {app.output_dict}")
+        except Empty:
+            pass
+
+
+class Payload(BaseModel):
+    text: str
+
+
+async def get_result(request_id):
+    """Get data from output dict that global."""
+    while True:
+        if request_id in app.output_dict:
+            result = app.output_dict[request_id]
+            del app.output_dict[request_id]
+            return {"message": result}
+        await asyncio.sleep(0.001)
+
+
+@app.post("/upper", response_class=UJSONResponse)
+async def message_upper(payload: Payload):
+    request_id: str = str(uuid.uuid4())
+    app.queue.put(
+        {"text": payload.text, "request_id": request_id},
+    )
+    return await get_result(request_id)
+
+
+if str2bool(os.getenv("WORKFLOW_API_ENABLE_ROUTE_WORKFLOW", "true")):
+    from .route import workflow
+
+    app.include_router(workflow)
+
+if str2bool(os.getenv("WORKFLOW_API_ENABLE_ROUTE_SCHEDULE", "true")):
+    from .route import schedule
+
+    app.include_router(schedule)