ddeutil-workflow 0.0.73__py3-none-any.whl → 0.0.75__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- ddeutil/workflow/__about__.py +1 -1
- ddeutil/workflow/__cron.py +20 -12
- ddeutil/workflow/__init__.py +119 -10
- ddeutil/workflow/__types.py +53 -41
- ddeutil/workflow/api/__init__.py +74 -3
- ddeutil/workflow/api/routes/job.py +15 -29
- ddeutil/workflow/api/routes/logs.py +9 -9
- ddeutil/workflow/api/routes/workflows.py +3 -3
- ddeutil/workflow/audits.py +70 -55
- ddeutil/workflow/cli.py +1 -15
- ddeutil/workflow/conf.py +71 -26
- ddeutil/workflow/errors.py +86 -19
- ddeutil/workflow/event.py +268 -169
- ddeutil/workflow/job.py +331 -192
- ddeutil/workflow/params.py +43 -11
- ddeutil/workflow/result.py +96 -70
- ddeutil/workflow/reusables.py +56 -6
- ddeutil/workflow/stages.py +1059 -572
- ddeutil/workflow/traces.py +205 -124
- ddeutil/workflow/utils.py +58 -19
- ddeutil/workflow/workflow.py +435 -296
- {ddeutil_workflow-0.0.73.dist-info → ddeutil_workflow-0.0.75.dist-info}/METADATA +27 -17
- ddeutil_workflow-0.0.75.dist-info/RECORD +30 -0
- ddeutil_workflow-0.0.73.dist-info/RECORD +0 -30
- {ddeutil_workflow-0.0.73.dist-info → ddeutil_workflow-0.0.75.dist-info}/WHEEL +0 -0
- {ddeutil_workflow-0.0.73.dist-info → ddeutil_workflow-0.0.75.dist-info}/entry_points.txt +0 -0
- {ddeutil_workflow-0.0.73.dist-info → ddeutil_workflow-0.0.75.dist-info}/licenses/LICENSE +0 -0
- {ddeutil_workflow-0.0.73.dist-info → ddeutil_workflow-0.0.75.dist-info}/top_level.txt +0 -0
@@ -1,6 +1,6 @@
|
|
1
1
|
Metadata-Version: 2.4
|
2
2
|
Name: ddeutil-workflow
|
3
|
-
Version: 0.0.
|
3
|
+
Version: 0.0.75
|
4
4
|
Summary: Lightweight workflow orchestration with YAML template
|
5
5
|
Author-email: ddeutils <korawich.anu@gmail.com>
|
6
6
|
License: MIT
|
@@ -68,6 +68,7 @@ by a `.yaml` template.
|
|
68
68
|
3. All parallel tasks inside workflow core engine use **Multi-Threading** pool
|
69
69
|
(Python 3.13 unlock GIL 🐍🔓)
|
70
70
|
4. Recommend to pass a **Secret Value** with environment variable in YAML template 🔐
|
71
|
+
5. Any datatime value convert to **UTC Timezone** 🌐
|
71
72
|
|
72
73
|
---
|
73
74
|
|
@@ -139,6 +140,15 @@ If you want to install this package with application add-ons, you should add
|
|
139
140
|
| Python | `ddeutil-workflow` | ✅ |
|
140
141
|
| FastAPI Server | `ddeutil-workflow[all]` | ✅ |
|
141
142
|
|
143
|
+
## 📖 Documentation
|
144
|
+
|
145
|
+
For comprehensive API documentation, examples, and best practices:
|
146
|
+
|
147
|
+
- **[Full Documentation](https://ddeutils.github.io/ddeutil-workflow/)** - Complete user guide and API reference
|
148
|
+
- **[Getting Started](https://ddeutils.github.io/ddeutil-workflow/getting-started/)** - Quick start guide
|
149
|
+
- **[API Reference](https://ddeutils.github.io/ddeutil-workflow/api/workflow/)** - Detailed API documentation
|
150
|
+
- **[Examples](https://ddeutils.github.io/ddeutil-workflow/examples/)** - Real-world usage examples
|
151
|
+
|
142
152
|
## 🎯 Usage
|
143
153
|
|
144
154
|
This is examples that use workflow file for running common Data Engineering
|
@@ -273,22 +283,22 @@ it will use default value and do not raise any error to you.
|
|
273
283
|
> The config value that you will set on the environment should combine with
|
274
284
|
> prefix, component, and name which is `WORKFLOW_{component}_{name}` (Upper case).
|
275
285
|
|
276
|
-
| Name
|
277
|
-
|
278
|
-
| **REGISTRY_CALLER**
|
279
|
-
| **REGISTRY_FILTER**
|
280
|
-
| **CONF_PATH**
|
281
|
-
| **
|
282
|
-
| **
|
283
|
-
| **
|
284
|
-
| **
|
285
|
-
| **FORMAT**
|
286
|
-
| **FORMAT_FILE**
|
287
|
-
| **DATETIME_FORMAT**
|
288
|
-
| **
|
289
|
-
| **TRACE_ENABLE_WRITE**
|
290
|
-
| **
|
291
|
-
| **AUDIT_ENABLE_WRITE**
|
286
|
+
| Name | Component | Default | Description |
|
287
|
+
|:----------------------------|:---------:|:--------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|
|
288
|
+
| **REGISTRY_CALLER** | CORE | `.` | List of importable string for the call stage. |
|
289
|
+
| **REGISTRY_FILTER** | CORE | `ddeutil.workflow.templates` | List of importable string for the filter template. |
|
290
|
+
| **CONF_PATH** | CORE | `./conf` | The config path that keep all template `.yaml` files. |
|
291
|
+
| **STAGE_DEFAULT_ID** | CORE | `false` | A flag that enable default stage ID that use for catch an execution output. |
|
292
|
+
| **GENERATE_ID_SIMPLE_MODE** | CORE | `true` | A flog that enable generating ID with `md5` algorithm. |
|
293
|
+
| **DEBUG_MODE** | LOG | `true` | A flag that enable logging with debug level mode. |
|
294
|
+
| **TIMEZONE** | LOG | `Asia/Bangkok` | A Timezone string value that will pass to `ZoneInfo` object. |
|
295
|
+
| **FORMAT** | LOG | `%(asctime)s.%(msecs)03d (%(name)-10s, %(process)-5d,%(thread)-5d) [%(levelname)-7s] %(message)-120s (%(filename)s:%(lineno)s)` | A trace message console format. |
|
296
|
+
| **FORMAT_FILE** | LOG | `{datetime} ({process:5d}, {thread:5d}) {message:120s} ({filename}:{lineno})` | A trace message format that use to write to target pointer. |
|
297
|
+
| **DATETIME_FORMAT** | LOG | `%Y-%m-%d %H:%M:%S` | A datetime format of the trace log. |
|
298
|
+
| **TRACE_URL** | LOG | `file:./logs` | A pointer URL of trace log that use to emit log message. |
|
299
|
+
| **TRACE_ENABLE_WRITE** | LOG | `false` | A flag that enable writing trace log. |
|
300
|
+
| **AUDIT_URL** | LOG | `file:./audits` | A pointer URL of audit log that use to write audit metrix. |
|
301
|
+
| **AUDIT_ENABLE_WRITE** | LOG | `true` | A flag that enable writing audit log after end execution in the workflow release step. |
|
292
302
|
|
293
303
|
## :rocket: Deployment
|
294
304
|
|
@@ -0,0 +1,30 @@
|
|
1
|
+
ddeutil/workflow/__about__.py,sha256=iaM2-rdtvV7sEgUAaF8bPdKBNJ3_zhFdOi5A1NC-s3U,28
|
2
|
+
ddeutil/workflow/__cron.py,sha256=0_40rdL3QEChMcikrLQfQPb67YfHmbYr_aPCp6NM7Cw,28912
|
3
|
+
ddeutil/workflow/__init__.py,sha256=kjKFdRNOh19IiLdmSFPKesU8BqSijEWHRZ8_fKGXFUk,3276
|
4
|
+
ddeutil/workflow/__main__.py,sha256=Qd-f8z2Q2vpiEP2x6PBFsJrpACWDVxFKQk820MhFmHo,59
|
5
|
+
ddeutil/workflow/__types.py,sha256=tA2vsr6mzTSzbWB1sb62c5GgxODlfVRz6FvgLNJtQao,4788
|
6
|
+
ddeutil/workflow/audits.py,sha256=PZ0dDBBANpXoLHlDrmlEoIXr0iYK9PCw4-9tPJX9UXE,12528
|
7
|
+
ddeutil/workflow/cli.py,sha256=V42C3lmtzRUiIjgMjLlr7v600PX0A3Aqbp8ccUSgQew,5244
|
8
|
+
ddeutil/workflow/conf.py,sha256=UCw6v2GFD3tA2LRbp7vLifXniey0P5Ef0U9eBPknrWk,16267
|
9
|
+
ddeutil/workflow/errors.py,sha256=n10YjXptY4iiY57FFVq22aedlUojpMuB5cM7aQ0KNpo,5522
|
10
|
+
ddeutil/workflow/event.py,sha256=siChcBhsu4ejzW1fK0tjHPXQVaSUCSxPYDgDrh6duwo,13676
|
11
|
+
ddeutil/workflow/job.py,sha256=oCjxG1Zz2D9Yc2SuZth3k1geIQ-AJ3oLs_38g5NMXLE,43994
|
12
|
+
ddeutil/workflow/params.py,sha256=Cyz142OcvENIZrM7Efc2xuGPmmFBhROifP5ojoaCezg,13658
|
13
|
+
ddeutil/workflow/result.py,sha256=LBzBdVlqXsG4vgHQadb_bfuxY32yQ-dlBf-HmnX-dBk,9135
|
14
|
+
ddeutil/workflow/reusables.py,sha256=q_OA-oifCGIhW_5j6hTZXZk7FBOmDt0xVrtNnscJfNg,23294
|
15
|
+
ddeutil/workflow/stages.py,sha256=2Rz9NGVOftIqiYaSgasHDS8MvgvLp3jtAjBHEwdtx7U,121837
|
16
|
+
ddeutil/workflow/traces.py,sha256=KFGeIFk1ocDWsaNkjXJk3LQHvq8dRfIf3hZsTuCpF1Y,28334
|
17
|
+
ddeutil/workflow/utils.py,sha256=EXhIuWzOJHvlcoAdyvuDUomGtMTIB59HxOLpj2VJ1bI,10857
|
18
|
+
ddeutil/workflow/workflow.py,sha256=izGz5teb0unlKtnpklvcdWFHK1af60IJVeSbUhMdPtA,41206
|
19
|
+
ddeutil/workflow/api/__init__.py,sha256=5DzYL3ngceoRshh5HYCSVWChqNJSiP01E1bEd8XxPi0,4799
|
20
|
+
ddeutil/workflow/api/log_conf.py,sha256=WfS3udDLSyrP-C80lWOvxxmhd_XWKvQPkwDqKblcH3E,1834
|
21
|
+
ddeutil/workflow/api/routes/__init__.py,sha256=JRaJZB0D6mgR17MbZo8yLtdYDtD62AA8MdKlFqhG84M,420
|
22
|
+
ddeutil/workflow/api/routes/job.py,sha256=-lbZ_hS9pEdSy6zeke5qrXEgdNxtQ2w9in7cHuM2Jzs,2536
|
23
|
+
ddeutil/workflow/api/routes/logs.py,sha256=RiZ62eQVMWArPHE3lpan955U4DdLLkethlvSMlwF7Mg,5312
|
24
|
+
ddeutil/workflow/api/routes/workflows.py,sha256=1Mqx4Hft4uJglgJI-Wcw-JzkhomFYZrtP0DnQDBkAFQ,4410
|
25
|
+
ddeutil_workflow-0.0.75.dist-info/licenses/LICENSE,sha256=nGFZ1QEhhhWeMHf9n99_fdt4vQaXS29xWKxt-OcLywk,1085
|
26
|
+
ddeutil_workflow-0.0.75.dist-info/METADATA,sha256=sJGNuv_qmZziEljC1T-9tCCOAGT3ovafMkAdRg-BvPo,15781
|
27
|
+
ddeutil_workflow-0.0.75.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
|
28
|
+
ddeutil_workflow-0.0.75.dist-info/entry_points.txt,sha256=qDTpPSauL0ciO6T4iSVt8bJeYrVEkkoEEw_RlGx6Kgk,63
|
29
|
+
ddeutil_workflow-0.0.75.dist-info/top_level.txt,sha256=m9M6XeSWDwt_yMsmH6gcOjHZVK5O0-vgtNBuncHjzW4,8
|
30
|
+
ddeutil_workflow-0.0.75.dist-info/RECORD,,
|
@@ -1,30 +0,0 @@
|
|
1
|
-
ddeutil/workflow/__about__.py,sha256=Ht-dVJZ9jffiZCn55sCsjQbRLgRrXbeX4g38xoyJcmo,28
|
2
|
-
ddeutil/workflow/__cron.py,sha256=BOKQcreiex0SAigrK1gnLxpvOeF3aca_rQwyz9Kfve4,28751
|
3
|
-
ddeutil/workflow/__init__.py,sha256=HUy9XkBe7ttpUupJS4JDuj3aGp2QmJZfz8m2kHAIwdw,927
|
4
|
-
ddeutil/workflow/__main__.py,sha256=Qd-f8z2Q2vpiEP2x6PBFsJrpACWDVxFKQk820MhFmHo,59
|
5
|
-
ddeutil/workflow/__types.py,sha256=uNfoRbVmNK5O37UUMVnqcmoghD9oMS1q9fXC0APnjSI,4584
|
6
|
-
ddeutil/workflow/audits.py,sha256=1pg4a5wdZCAKOqMr1Z_ofzRAFsDarN1BIJenWwn9xkg,11435
|
7
|
-
ddeutil/workflow/cli.py,sha256=7cdn4f9KYOBK4CFs7sGmM2c0iYvzXiPtHnklSuD7ObU,5676
|
8
|
-
ddeutil/workflow/conf.py,sha256=dxPzaLL9Wsg_3k6diQn4jqxT9jkqdsjDEGl81qLhVpY,14472
|
9
|
-
ddeutil/workflow/errors.py,sha256=O5rq80Sqj0QMeIsWXpRUhiFLTq0o8bwm5BQ4kuq6xmI,3013
|
10
|
-
ddeutil/workflow/event.py,sha256=0fGcous2vAF5DZ-aengk_vovgHoWNRoIiidSxk9ytkw,11119
|
11
|
-
ddeutil/workflow/job.py,sha256=kviOQeSUsx0Z7CL0foblTulg2m_l6a3M3SMRxg9RWeg,39151
|
12
|
-
ddeutil/workflow/params.py,sha256=Pco3DyjptC5Jkx53dhLL9xlIQdJvNAZs4FLzMUfXpbQ,12402
|
13
|
-
ddeutil/workflow/result.py,sha256=ctxNSaY9tZPHEAUgvDkjWWu2APeTmlZCf1Hb0XVbbFo,8173
|
14
|
-
ddeutil/workflow/reusables.py,sha256=LSn0XTkzGHf4ulOmWub29F0JZHt0NEyzrFd4ZFx_g_k,21622
|
15
|
-
ddeutil/workflow/stages.py,sha256=xJqVxJvah3nsJSgOHL3BFm5M_UzM8RSEeIIlx96qUAI,107564
|
16
|
-
ddeutil/workflow/traces.py,sha256=2XTX0ZQnbtKAp__PU4KCuBrx_8fTDpiIiBnKqeFCrLY,25318
|
17
|
-
ddeutil/workflow/utils.py,sha256=-yWVShFGUJwUL5DU4xi-f2UGLXEuuv8sTf-GmVGjNt0,9674
|
18
|
-
ddeutil/workflow/workflow.py,sha256=sy-HEKLmGPE1tDNr_cpm3CQsMEDP6fkXDHC9UsVSVDE,36214
|
19
|
-
ddeutil/workflow/api/__init__.py,sha256=W3fe6_NLHSUzr4Tsu79w3pmvrYjpLeP3zBk4mtpPyqg,2843
|
20
|
-
ddeutil/workflow/api/log_conf.py,sha256=WfS3udDLSyrP-C80lWOvxxmhd_XWKvQPkwDqKblcH3E,1834
|
21
|
-
ddeutil/workflow/api/routes/__init__.py,sha256=JRaJZB0D6mgR17MbZo8yLtdYDtD62AA8MdKlFqhG84M,420
|
22
|
-
ddeutil/workflow/api/routes/job.py,sha256=x809G5gCbJS257txj9eLLTbCbFK8ercXWzPDLuv5gEM,2953
|
23
|
-
ddeutil/workflow/api/routes/logs.py,sha256=HiXw93PeIiaK_xJjM8lbD2ED1Il-W1iM51085nc7qmg,5286
|
24
|
-
ddeutil/workflow/api/routes/workflows.py,sha256=D76cdLb2_9Dkfe2_8xt06CvPhAyJMqxyYkUgAs8Qlnw,4402
|
25
|
-
ddeutil_workflow-0.0.73.dist-info/licenses/LICENSE,sha256=nGFZ1QEhhhWeMHf9n99_fdt4vQaXS29xWKxt-OcLywk,1085
|
26
|
-
ddeutil_workflow-0.0.73.dist-info/METADATA,sha256=0R6aQshHh_MWHRGKQwlrCfUToucyG3cLRuEDv_QDTKY,15221
|
27
|
-
ddeutil_workflow-0.0.73.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
|
28
|
-
ddeutil_workflow-0.0.73.dist-info/entry_points.txt,sha256=qDTpPSauL0ciO6T4iSVt8bJeYrVEkkoEEw_RlGx6Kgk,63
|
29
|
-
ddeutil_workflow-0.0.73.dist-info/top_level.txt,sha256=m9M6XeSWDwt_yMsmH6gcOjHZVK5O0-vgtNBuncHjzW4,8
|
30
|
-
ddeutil_workflow-0.0.73.dist-info/RECORD,,
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|