ddeutil-workflow 0.0.7__py3-none-any.whl → 0.0.8__py3-none-any.whl

{ddeutil_workflow-0.0.7.dist-info → ddeutil_workflow-0.0.8.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: ddeutil-workflow
-Version: 0.0.7
+Version: 0.0.8
 Summary: Data Developer & Engineer Workflow Utility Objects
 Author-email: ddeutils <korawich.anu@gmail.com>
 License: MIT
@@ -79,13 +79,10 @@ pip install ddeutil-workflow
 This project need `ddeutil-io` extension namespace packages. If you want to install
 this package with application add-ons, you should add `app` in installation;
 
-```shell
-pip install ddeutil-workflow[app]
-```
-
-```shell
-pip install ddeutil-workflow[api]
-```
+| Usecase            | Install Optional          |
+|--------------------|---------------------------|
+| Scheduler Service  | `ddeutil-workflow[app]`   |
+| FastAPI Server     | `ddeutil-workflow[api]`   |
 
 ## Getting Started
 
@@ -168,11 +165,12 @@ use-case.
 > I recommend you to use `task` stage for all actions that you want to do with
 > pipeline object.
 
-### Python & Bash
-
 ```yaml
 run_py_local:
   type: pipeline.Pipeline
+  on:
+    - cronjob: '* * * * *'
+      timezone: "Asia/Bangkok"
   params:
     author-run: str
     run-date: datetime
@@ -182,12 +180,11 @@ run_py_local:
         - name: "Printing Information"
           id: define-func
           run: |
-            x = '${{ params.author-run }}'
-            print(f'Hello {x}')
+            x = '${{ params.run-date | fmt("%Y%m%d") }}'
+            print(f'Hello at {x}')
 
             def echo(name: str):
               print(f'Hello {name}')
-
         - name: "Run Sequence and use var from Above"
           vars:
             x: ${{ params.author-run }}
@@ -195,7 +192,6 @@ run_py_local:
             print(f'Receive x from above with {x}')
             # Change x value
             x: int = 1
-
         - name: "Call Function"
           vars:
             echo: ${{ stages.define-func.outputs.echo }}
@@ -210,93 +206,23 @@ run_py_local:
 ```
 
 ```python
+from datetime import datetime
 from ddeutil.workflow.pipeline import Pipeline
 
-pipe = Pipeline.from_loader(name='run_py_local', externals={})
-pipe.execute(params={'author-run': 'Local Workflow', 'run-date': '2024-01-01'})
+pipe: Pipeline = Pipeline.from_loader(name='run_py_local', externals={})
+pipe.execute(params={
+    'author-run': 'Local Workflow',
+    'run-date': datetime(2024, 1, 1),
+})
 ```
 
 ```shell
-> Hello Local Workflow
+> Hello at 20240101
 > Receive x from above with Local Workflow
 > Hello Caller
 > Hello World from Shell
 ```
 
-### Hook (Extract & Load)
-
-```yaml
-pipe_el_pg_to_lake:
-  type: pipeline.Pipeline
-  params:
-    run-date: datetime
-    author-email: str
-  jobs:
-    extract-load:
-      stages:
-        - name: "Extract Load from Postgres to Lake"
-          id: extract-load
-          uses: tasks/postgres-to-delta@polars
-          with:
-            source:
-              conn: conn_postgres_url
-              query: |
-                select * from ${{ params.name }}
-                where update_date = '${{ params.datetime }}'
-            sink:
-              conn: conn_az_lake
-              endpoint: "/${{ params.name }}"
-```
-
-Implement hook:
-
-```python
-from ddeutil.workflow.utils import tag
-
-@tag('polars', alias='postgres-to-delta')
-def postgres_to_delta(source, sink):
-    return {
-        "source": source, "sink": sink
-    }
-```
-
-### Hook (Transform)
-
-```yaml
-pipeline_hook_mssql_proc:
-  type: pipeline.Pipeline
-  params:
-    run_date: datetime
-    sp_name: str
-    source_name: str
-    target_name: str
-  jobs:
-    transform:
-      stages:
-        - name: "Transform Data in MS SQL Server"
-          id: transform
-          uses: tasks/mssql-proc@odbc
-          with:
-            exec: ${{ params.sp_name }}
-            params:
-              run_mode: "T"
-              run_date: ${{ params.run_date }}
-              source: ${{ params.source_name }}
-              target: ${{ params.target_name }}
-```
-
-Implement hook:
-
-```python
-from ddeutil.workflow.utils import tag
-
-@tag('odbc', alias='mssql-proc')
-def odbc_mssql_procedure(_exec: str, params: dict):
-    return {
-        "exec": _exec, "params": params
-    }
-```
-
 ## Configuration
 
 ```bash
@@ -306,7 +232,6 @@ export WORKFLOW_CORE_REGISTRY_FILTER=ddeutil.workflow.utils
 export WORKFLOW_CORE_PATH_CONF=conf
 export WORKFLOW_CORE_TIMEZONE=Asia/Bangkok
 export WORKFLOW_CORE_DEFAULT_STAGE_ID=true
-
 export WORKFLOW_CORE_MAX_PIPELINE_POKING=4
 export WORKFLOW_CORE_MAX_JOB_PARALLEL=2
 ```

ddeutil_workflow-0.0.8.dist-info/RECORD

@@ -0,0 +1,20 @@
+ddeutil/workflow/__about__.py,sha256=FA15NQYpQvn7SrHupxQQQ9Ad5ZzEXOvwDS5UyB1h1bo,27
+ddeutil/workflow/__init__.py,sha256=4PEL3RdHmUowK0Dz-tK7fO0wvFX4u9CLd0Up7b3lrAQ,760
+ddeutil/workflow/__types.py,sha256=SYMoxbENQX8uPsiCZkjtpHAqqHOh8rUrarAFicAJd0E,1773
+ddeutil/workflow/api.py,sha256=d2Mmv9jTtN3FITIy-2mivyAKdBOGZxtkNWRMPbCLlFI,3341
+ddeutil/workflow/app.py,sha256=BuYhOoSJCHiSoj3xb2I5QoxaHrD3bKdmoJua3bKBetc,1165
+ddeutil/workflow/exceptions.py,sha256=zuCcsfJ1hFivubXz6lXCpGYXk07d_PkRaUD5ew3_LC0,632
+ddeutil/workflow/loader.py,sha256=_ZD-XP5P7VbUeqItrUVPaKIZu6dMUZ2aywbCbReW1hQ,2778
+ddeutil/workflow/log.py,sha256=N2TyjcuAoH0YTvzJCHTO037IHgVkLA986Xhtz1LSgE4,1742
+ddeutil/workflow/on.py,sha256=YoEqDbzJUwqOA3JRltbvlYr0rNTtxdmb7cWMxl8U19k,6717
+ddeutil/workflow/pipeline.py,sha256=VC6VDxycUdGKn13V42RZxAlCFySYb2HIZGq_ku5Kp5k,30844
+ddeutil/workflow/repeat.py,sha256=sNoRfbOR4cYm_edrSvlVy9N8Dk_osLIq9FC5GMZz32M,4621
+ddeutil/workflow/route.py,sha256=Ck_O1xJwI-vKkMJr37El0-1PGKlwKF8__DDNWVQrf0A,2079
+ddeutil/workflow/scheduler.py,sha256=FqmkvWCqwJ4eRf8aDn5Ce4FcNWqmcvu2aTTfL34lfgs,22184
+ddeutil/workflow/stage.py,sha256=tbxENx_-2BQ6peXKM_s6RQ1oGzTlXcZ4yDpP1Hufkdk,18095
+ddeutil/workflow/utils.py,sha256=seyU81JXfb2zz6QbJvVEb2Wn4qt8f-FBA6QFC97xY5k,21240
+ddeutil_workflow-0.0.8.dist-info/LICENSE,sha256=nGFZ1QEhhhWeMHf9n99_fdt4vQaXS29xWKxt-OcLywk,1085
+ddeutil_workflow-0.0.8.dist-info/METADATA,sha256=9i7Jk3CZlBpNkmFFjD247opgYA6Mc8AT6CtZjcvamYI,8314
+ddeutil_workflow-0.0.8.dist-info/WHEEL,sha256=HiCZjzuy6Dw0hdX5R3LCFPDmFS4BWl8H-8W39XfmgX4,91
+ddeutil_workflow-0.0.8.dist-info/top_level.txt,sha256=m9M6XeSWDwt_yMsmH6gcOjHZVK5O0-vgtNBuncHjzW4,8
+ddeutil_workflow-0.0.8.dist-info/RECORD,,

ddeutil_workflow-0.0.7.dist-info/RECORD

@@ -1,20 +0,0 @@
-ddeutil/workflow/__about__.py,sha256=b23XabBwtuoPOLmS_Hj_gSA4LZ0fRfAkACM6c3szVoc,27
-ddeutil/workflow/__init__.py,sha256=4PEL3RdHmUowK0Dz-tK7fO0wvFX4u9CLd0Up7b3lrAQ,760
-ddeutil/workflow/__types.py,sha256=SYMoxbENQX8uPsiCZkjtpHAqqHOh8rUrarAFicAJd0E,1773
-ddeutil/workflow/api.py,sha256=d2Mmv9jTtN3FITIy-2mivyAKdBOGZxtkNWRMPbCLlFI,3341
-ddeutil/workflow/app.py,sha256=GbdwvUkE8lO2Ze4pZ0-J-7p9mcZAaORfjkHwW_oZIP0,1076
-ddeutil/workflow/exceptions.py,sha256=BH7COn_3uz3z7oJBZOQGiuo8osBFgeXL8HYymnjCOPQ,671
-ddeutil/workflow/loader.py,sha256=_ZD-XP5P7VbUeqItrUVPaKIZu6dMUZ2aywbCbReW1hQ,2778
-ddeutil/workflow/log.py,sha256=_GJEdJr7bqpcQDxZjrqHd-hkiW3NKFaVoR6voE6Ty0o,952
-ddeutil/workflow/on.py,sha256=YoEqDbzJUwqOA3JRltbvlYr0rNTtxdmb7cWMxl8U19k,6717
-ddeutil/workflow/pipeline.py,sha256=dKF09TFS_v5TCD-5o8tp1UhB5sGuWIQu4zl_UFtlIC0,25951
-ddeutil/workflow/repeat.py,sha256=sNoRfbOR4cYm_edrSvlVy9N8Dk_osLIq9FC5GMZz32M,4621
-ddeutil/workflow/route.py,sha256=Ck_O1xJwI-vKkMJr37El0-1PGKlwKF8__DDNWVQrf0A,2079
-ddeutil/workflow/scheduler.py,sha256=FqmkvWCqwJ4eRf8aDn5Ce4FcNWqmcvu2aTTfL34lfgs,22184
-ddeutil/workflow/stage.py,sha256=z05bKk2QFQDXjidSnQYCVOdceSpSO13sHXE0B1UH6XA,14978
-ddeutil/workflow/utils.py,sha256=pDM2jaYVP-USH0pLd_XmHOguxVPGVzZ76hOh1AZdINU,18495
-ddeutil_workflow-0.0.7.dist-info/LICENSE,sha256=nGFZ1QEhhhWeMHf9n99_fdt4vQaXS29xWKxt-OcLywk,1085
-ddeutil_workflow-0.0.7.dist-info/METADATA,sha256=ba2nH57cpHB2P4ldQCRT8ZWDj3r1OPx9a1dgcB0a2Ws,9702
-ddeutil_workflow-0.0.7.dist-info/WHEEL,sha256=HiCZjzuy6Dw0hdX5R3LCFPDmFS4BWl8H-8W39XfmgX4,91
-ddeutil_workflow-0.0.7.dist-info/top_level.txt,sha256=m9M6XeSWDwt_yMsmH6gcOjHZVK5O0-vgtNBuncHjzW4,8
-ddeutil_workflow-0.0.7.dist-info/RECORD,,