dotflow 0.15.0.dev1__tar.gz → 0.15.0.dev3__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (110) hide show
  1. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/PKG-INFO +189 -208
  2. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/README.md +183 -207
  3. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/__init__.py +1 -1
  4. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/abc/flow.py +19 -0
  5. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cli/commands/cloud.py +8 -1
  6. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cli/commands/deploy.py +35 -3
  7. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cli/commands/init.py +1 -3
  8. dotflow-0.15.0.dev3/dotflow/cli/commands/storage.py +55 -0
  9. dotflow-0.15.0.dev3/dotflow/cloud/aws/__init__.py +15 -0
  10. dotflow-0.15.0.dev1/dotflow/cloud/aws/lambda_deployer.py → dotflow-0.15.0.dev3/dotflow/cloud/aws/base_lambda_deployer.py +30 -51
  11. dotflow-0.15.0.dev3/dotflow/cloud/aws/lambda_api_deployer.py +67 -0
  12. dotflow-0.15.0.dev3/dotflow/cloud/aws/lambda_deployer.py +49 -0
  13. dotflow-0.15.0.dev3/dotflow/cloud/aws/lambda_s3_deployer.py +56 -0
  14. dotflow-0.15.0.dev3/dotflow/cloud/aws/lambda_sqs_deployer.py +43 -0
  15. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/core.py +22 -0
  16. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/gcp/services/apis.py +5 -3
  17. dotflow-0.15.0.dev3/dotflow/cloud/github/__init__.py +5 -0
  18. dotflow-0.15.0.dev3/dotflow/cloud/github/actions_deployer.py +116 -0
  19. dotflow-0.15.0.dev3/dotflow/cloud/github/constants.py +8 -0
  20. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/action.py +40 -79
  21. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/context.py +17 -4
  22. dotflow-0.15.0.dev3/dotflow/core/engine.py +277 -0
  23. dotflow-0.15.0.dev3/dotflow/core/module.py +30 -0
  24. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/serializers/task.py +7 -9
  25. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/workflow.py +46 -62
  26. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/__init__.py +12 -0
  27. dotflow-0.15.0.dev3/dotflow/providers/log_sentry.py +96 -0
  28. dotflow-0.15.0.dev3/dotflow/providers/storage_default.py +22 -0
  29. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/storage_file.py +11 -8
  30. dotflow-0.15.0.dev3/dotflow/providers/tracer_sentry.py +126 -0
  31. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/settings.py +1 -1
  32. dotflow-0.15.0.dev3/dotflow/utils/error_handler.py +13 -0
  33. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/utils/tools.py +0 -10
  34. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/pyproject.toml +4 -2
  35. dotflow-0.15.0.dev1/dotflow/cloud/aws/__init__.py +0 -6
  36. dotflow-0.15.0.dev1/dotflow/core/execution.py +0 -176
  37. dotflow-0.15.0.dev1/dotflow/core/module.py +0 -36
  38. dotflow-0.15.0.dev1/dotflow/providers/storage_default.py +0 -20
  39. dotflow-0.15.0.dev1/dotflow/storage.py +0 -22
  40. dotflow-0.15.0.dev1/dotflow/utils/error_handler.py +0 -22
  41. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/LICENSE +0 -0
  42. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/abc/__init__.py +0 -0
  43. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/abc/api.py +0 -0
  44. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/abc/file.py +0 -0
  45. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/abc/http.py +0 -0
  46. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/abc/log.py +0 -0
  47. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/abc/metrics.py +0 -0
  48. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/abc/notify.py +0 -0
  49. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/abc/scheduler.py +0 -0
  50. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/abc/storage.py +0 -0
  51. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/abc/tcp.py +0 -0
  52. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/abc/tracer.py +0 -0
  53. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cli/__init__.py +0 -0
  54. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cli/command.py +0 -0
  55. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cli/commands/__init__.py +0 -0
  56. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cli/commands/log.py +0 -0
  57. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cli/commands/schedule.py +0 -0
  58. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cli/commands/start.py +0 -0
  59. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cli/setup.py +0 -0
  60. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cli/validators/__init__.py +0 -0
  61. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cli/validators/start.py +0 -0
  62. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/__init__.py +0 -0
  63. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/aws/constants.py +0 -0
  64. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/aws/ecs_deployer.py +0 -0
  65. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/aws/services/__init__.py +0 -0
  66. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/aws/services/cloudwatch.py +0 -0
  67. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/aws/services/ecr.py +0 -0
  68. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/aws/services/iam.py +0 -0
  69. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/aws/services/s3.py +0 -0
  70. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/gcp/__init__.py +0 -0
  71. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/gcp/cloudrun_deployer.py +0 -0
  72. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/gcp/constants.py +0 -0
  73. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/gcp/services/__init__.py +0 -0
  74. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/gcp/services/artifact_registry.py +0 -0
  75. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/cloud/gcp/services/gcs.py +0 -0
  76. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/__init__.py +0 -0
  77. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/config.py +0 -0
  78. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/decorators/__init__.py +0 -0
  79. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/decorators/time.py +0 -0
  80. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/dotflow.py +0 -0
  81. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/exception.py +0 -0
  82. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/serializers/__init__.py +0 -0
  83. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/serializers/transport.py +0 -0
  84. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/serializers/workflow.py +0 -0
  85. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/task.py +0 -0
  86. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/types/__init__.py +0 -0
  87. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/types/execution.py +0 -0
  88. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/types/overlap.py +0 -0
  89. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/types/status.py +0 -0
  90. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/types/storage.py +0 -0
  91. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/core/types/workflow.py +0 -0
  92. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/logging.py +0 -0
  93. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/main.py +0 -0
  94. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/api_default.py +0 -0
  95. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/log_default.py +0 -0
  96. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/log_opentelemetry.py +0 -0
  97. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/metrics_default.py +0 -0
  98. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/metrics_opentelemetry.py +0 -0
  99. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/notify_default.py +0 -0
  100. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/notify_discord.py +0 -0
  101. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/notify_telegram.py +0 -0
  102. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/scheduler_cron.py +0 -0
  103. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/scheduler_default.py +0 -0
  104. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/storage_gcs.py +0 -0
  105. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/storage_s3.py +0 -0
  106. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/tracer_default.py +0 -0
  107. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/providers/tracer_opentelemetry.py +0 -0
  108. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/types.py +0 -0
  109. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/utils/__init__.py +0 -0
  110. {dotflow-0.15.0.dev1 → dotflow-0.15.0.dev3}/dotflow/utils/basic_functions.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: dotflow
3
- Version: 0.15.0.dev1
3
+ Version: 0.15.0.dev3
4
4
  Summary: 🎲 Dotflow turns an idea into flow! Lightweight Python library for execution pipelines with retry, parallel, cron and async support.
5
5
  License: MIT License
6
6
 
@@ -44,9 +44,13 @@ Classifier: Programming Language :: Python :: 3.13
44
44
  Provides-Extra: aws
45
45
  Provides-Extra: deploy-aws
46
46
  Provides-Extra: deploy-gcp
47
+ Provides-Extra: deploy-github
47
48
  Provides-Extra: gcp
48
49
  Provides-Extra: otel
49
50
  Provides-Extra: scheduler
51
+ Provides-Extra: sentry
52
+ Requires-Dist: GitPython ; extra == "deploy-github"
53
+ Requires-Dist: PyGithub ; extra == "deploy-github"
50
54
  Requires-Dist: boto3 ; extra == "aws"
51
55
  Requires-Dist: boto3 ; extra == "deploy-aws"
52
56
  Requires-Dist: cookiecutter (>=2.0)
@@ -63,6 +67,7 @@ Requires-Dist: opentelemetry-sdk ; extra == "otel"
63
67
  Requires-Dist: pydantic
64
68
  Requires-Dist: requests
65
69
  Requires-Dist: rich
70
+ Requires-Dist: sentry-sdk ; extra == "sentry"
66
71
  Requires-Dist: typing-extensions
67
72
  Project-URL: Changelog, https://dotflow-io.github.io/dotflow/nav/development/release-notes/
68
73
  Project-URL: Documentation, https://dotflow-io.github.io/dotflow/
@@ -72,86 +77,40 @@ Project-URL: Repository, https://github.com/dotflow-io/dotflow
72
77
  Description-Content-Type: text/markdown
73
78
 
74
79
  <div align="center">
75
- <a aria-label="Serverless.com" href="https://dotflow.io">Website</a>
76
- &nbsp;•&nbsp;
77
- <a aria-label="Dotflow Documentation" href="https://dotflow-io.github.io/dotflow/">Documentation</a>
78
- &nbsp;•&nbsp;
79
- <a aria-label="Pypi" href="https://pypi.org/project/dotflow/">Pypi</a>
80
- </div>
81
80
 
82
- <br/>
81
+ ![](docs/assets/dotflow.png)
83
82
 
84
- <div align="center">
83
+ **Build production-ready data pipelines in minutes, not days.**
85
84
 
86
- ![](https://raw.githubusercontent.com/FernandoCelmer/dotflow/master/docs/assets/dotflow.gif)
85
+ [![PyPI](https://img.shields.io/pypi/v/dotflow?style=flat-square)](https://pypi.org/project/dotflow/)
86
+ [![Python](https://img.shields.io/pypi/pyversions/dotflow?style=flat-square)](https://pypi.org/project/dotflow/)
87
+ [![Downloads](https://img.shields.io/pypi/dm/dotflow?style=flat-square)](https://pypi.org/project/dotflow/)
88
+ [![Stars](https://img.shields.io/github/stars/dotflow-io?label=Stars&style=flat-square)](https://github.com/dotflow-io/dotflow)
87
89
 
88
- ![GitHub Org's stars](https://img.shields.io/github/stars/dotflow-io?label=Dotflow&style=flat-square)
89
- ![GitHub last commit](https://img.shields.io/github/last-commit/dotflow-io/dotflow?style=flat-square)
90
- ![PyPI](https://img.shields.io/pypi/v/dotflow?style=flat-square)
91
- ![PyPI - Python Version](https://img.shields.io/pypi/pyversions/dotflow?style=flat-square)
92
- ![PyPI - Downloads](https://img.shields.io/pypi/dm/dotflow?style=flat-square)
90
+ [Website](https://dotflow.io) · [Documentation](https://dotflow-io.github.io/dotflow/) · [PyPI](https://pypi.org/project/dotflow/)
93
91
 
94
92
  </div>
95
93
 
96
- # Welcome to Dotflow
97
-
98
- Dotflow is a lightweight Python library for building execution pipelines. Define tasks with decorators, chain them together, and run workflows in sequential, parallel, or background mode — with built-in retry, timeout, storage, notifications, and more.
99
-
100
- > **[Read the full documentation](https://dotflow-io.github.io/dotflow/)**
101
-
102
- ## Table of Contents
103
-
104
- <details>
105
- <summary>Click to expand</summary>
106
-
107
- - [Getting Help](#getting-help)
108
- - [Installation](#installation)
109
- - [Quick Start](#quick-start)
110
- - [Features](#features)
111
- - [Execution Modes](#execution-modes)
112
- - [Retry, Timeout & Backoff](#retry-timeout--backoff)
113
- - [Context System](#context-system)
114
- - [Checkpoint & Resume](#checkpoint--resume)
115
- - [Storage Providers](#storage-providers)
116
- - [Notifications](#notifications)
117
- - [Class-Based Steps](#class-based-steps)
118
- - [Task Groups](#task-groups)
119
- - [Callbacks](#callbacks)
120
- - [Error Handling](#error-handling)
121
- - [Async Support](#async-support)
122
- - [Scheduler / Cron](#scheduler--cron)
123
- - [CLI](#cli)
124
- - [Dependency Injection via Config](#dependency-injection-via-config)
125
- - [More Examples](#more-examples)
126
- - [Commit Style](#commit-style)
127
- - [License](#license)
94
+ ---
128
95
 
129
- </details>
96
+ # Dotflow
130
97
 
131
- ## Getting Help
98
+ Dotflow is a lightweight Python library for execution pipelines. Define tasks with decorators, chain them together, and deploy to any cloud — with built-in retry, parallel execution, storage, observability, and cloud deployment.
132
99
 
133
- We use GitHub issues for tracking bugs and feature requests.
100
+ ## Why Dotflow?
134
101
 
135
- - [Bug Report](https://github.com/dotflow-io/dotflow/issues/new/choose)
136
- - [Documentation](https://github.com/dotflow-io/dotflow/issues/new/choose)
137
- - [Feature Request](https://github.com/dotflow-io/dotflow/issues/new/choose)
138
- - [Security Issue](https://github.com/dotflow-io/dotflow/issues/new/choose)
139
- - [General Question](https://github.com/dotflow-io/dotflow/issues/new/choose)
102
+ - **Simple** — `@action` decorator + `workflow.start()`. That's it.
103
+ - **Resilient** — Retry, backoff, timeout, checkpoints, and error handling out of the box.
104
+ - **Observable** — OpenTelemetry traces, metrics, and logs. Sentry error tracking.
105
+ - **Deployable** — `dotflow deploy --platform lambda` ships your pipeline to AWS in one command.
106
+ - **Portable** — Same code runs on Lambda, ECS, Cloud Run, Kubernetes, Docker, or GitHub Actions.
140
107
 
141
- ## Installation
108
+ ## Install
142
109
 
143
110
  ```bash
144
111
  pip install dotflow
145
112
  ```
146
113
 
147
- **Optional extras:**
148
-
149
- ```bash
150
- pip install dotflow[aws] # AWS S3 storage
151
- pip install dotflow[gcp] # Google Cloud Storage
152
- pip install dotflow[scheduler] # Cron-based scheduler
153
- ```
154
-
155
114
  ## Quick Start
156
115
 
157
116
  ```python
@@ -178,9 +137,94 @@ workflow.task.add(step=load)
178
137
  workflow.start()
179
138
  ```
180
139
 
140
+ ## Deploy anywhere
141
+
142
+ Write your pipeline once. Deploy to any cloud with a single command.
143
+
144
+ ```bash
145
+ dotflow init
146
+ dotflow deploy --platform lambda --project my_pipeline
147
+ ```
148
+
149
+ ### Supported platforms
150
+
151
+ | Platform | Deploy method |
152
+ |----------|---------------|
153
+ | <img src="https://cdn.simpleicons.org/docker" width="16" /> **Docker** | `docker compose up` |
154
+ | <img src="https://www.vectorlogo.zone/logos/amazon_awslambda/amazon_awslambda-icon.svg" width="16" /> **AWS Lambda** | `dotflow deploy` |
155
+ | <img src="https://www.vectorlogo.zone/logos/amazon_awslambda/amazon_awslambda-icon.svg" width="16" /> **AWS Lambda + EventBridge** | `dotflow deploy --schedule` |
156
+ | <img src="https://www.vectorlogo.zone/logos/amazon_awslambda/amazon_awslambda-icon.svg" width="16" /> **AWS Lambda + S3 Trigger** | `dotflow deploy` |
157
+ | <img src="https://www.vectorlogo.zone/logos/amazon_awslambda/amazon_awslambda-icon.svg" width="16" /> **AWS Lambda + SQS Trigger** | `dotflow deploy` |
158
+ | <img src="https://www.vectorlogo.zone/logos/amazon_awslambda/amazon_awslambda-icon.svg" width="16" /> **AWS Lambda + API Gateway** | `dotflow deploy` |
159
+ | <img src="https://www.vectorlogo.zone/logos/amazon_ecs/amazon_ecs-icon.svg" width="16" /> **AWS ECS Fargate** | `dotflow deploy` |
160
+ | <img src="https://cdn.simpleicons.org/googlecloud" width="16" /> **Google Cloud Run** | `gcloud run deploy` |
161
+ | <img src="https://cdn.simpleicons.org/kubernetes" width="16" /> **Kubernetes** | `kubectl apply` |
162
+ | <img src="https://cdn.simpleicons.org/githubactions" width="16" /> **GitHub Actions** | `dotflow deploy` |
163
+
164
+ > [See all 34+ platforms →](https://dotflow-io.github.io/dotflow/nav/cloud/)
165
+
166
+ ## Optional extras
167
+
168
+ ```bash
169
+ pip install dotflow[aws] # S3 storage
170
+ pip install dotflow[gcp] # Google Cloud Storage
171
+ pip install dotflow[scheduler] # Cron scheduler
172
+ pip install dotflow[otel] # OpenTelemetry
173
+ pip install dotflow[sentry] # Sentry error tracking
174
+ pip install dotflow[deploy-aws] # AWS deploy (Lambda, ECS)
175
+ pip install dotflow[deploy-gcp] # GCP deploy (Cloud Run)
176
+ pip install dotflow[deploy-github] # GitHub Actions deploy
177
+ ```
178
+
179
+ > **[Read the full documentation →](https://dotflow-io.github.io/dotflow/)**
180
+
181
+ ## Documentation
182
+
183
+ | Section | Description |
184
+ |---------|-------------|
185
+ | [Concepts](https://dotflow-io.github.io/dotflow/nav/concepts/concept-workflow-and-tasks/) | Workflows, tasks, context, providers, process modes |
186
+ | [How-to Guides](https://dotflow-io.github.io/dotflow/nav/how-to/) | Step-by-step tutorials for workflows, tasks, and CLI |
187
+ | [Cloud Deployment](https://dotflow-io.github.io/dotflow/nav/cloud/) | Deploy to AWS, GCP, Kubernetes, Docker, GitHub Actions |
188
+ | [Integrations](https://dotflow-io.github.io/dotflow/nav/integrations/) | OpenTelemetry, Sentry, Telegram, Discord, S3, GCS |
189
+ | [Examples](https://dotflow-io.github.io/dotflow/nav/examples/) | Real-world pipelines: ETL, health checks, async, scheduler |
190
+ | [Reference](https://dotflow-io.github.io/dotflow/nav/reference/dotflow/) | API reference for all classes and providers |
191
+ | [Custom Providers](https://dotflow-io.github.io/dotflow/nav/development/custom-providers/) | Build your own storage, notify, log, tracer, or metrics provider |
192
+
181
193
  ## Features
182
194
 
183
- ### Execution Modes
195
+ <details>
196
+ <summary><strong>Observability</strong></summary>
197
+
198
+ > [OpenTelemetry docs](https://dotflow-io.github.io/dotflow/nav/tutorial/log-opentelemetry/) | [Sentry docs](https://dotflow-io.github.io/dotflow/nav/tutorial/log-sentry/) | [Tracer docs](https://dotflow-io.github.io/dotflow/nav/tutorial/tracer-opentelemetry/) | [Metrics docs](https://dotflow-io.github.io/dotflow/nav/tutorial/metrics-opentelemetry/)
199
+
200
+ Built-in support for OpenTelemetry and Sentry:
201
+
202
+ ```python
203
+ from dotflow import Config
204
+ from dotflow.providers import LogOpenTelemetry, TracerOpenTelemetry, MetricsOpenTelemetry
205
+
206
+ # OpenTelemetry: traces, metrics, and structured logs
207
+ config = Config(
208
+ log=LogOpenTelemetry(service_name="my-pipeline"),
209
+ tracer=TracerOpenTelemetry(service_name="my-pipeline"),
210
+ metrics=MetricsOpenTelemetry(service_name="my-pipeline"),
211
+ )
212
+ ```
213
+
214
+ ```python
215
+ from dotflow.providers import LogSentry, TracerSentry
216
+
217
+ # Sentry: error tracking + performance monitoring
218
+ config = Config(
219
+ log=LogSentry(dsn="https://xxx@sentry.io/123"),
220
+ tracer=TracerSentry(),
221
+ )
222
+ ```
223
+
224
+ </details>
225
+
226
+ <details>
227
+ <summary><strong>Execution Modes</strong></summary>
184
228
 
185
229
  > [Process Mode docs](https://dotflow-io.github.io/dotflow/nav/concepts/process-mode-sequential/)
186
230
 
@@ -251,7 +295,10 @@ flowchart TD
251
295
 
252
296
  ---
253
297
 
254
- ### Retry, Timeout & Backoff
298
+ </details>
299
+
300
+ <details>
301
+ <summary><strong>Retry, Timeout & Backoff</strong></summary>
255
302
 
256
303
  > [Retry docs](https://dotflow-io.github.io/dotflow/nav/tutorial/task-retry/) | [Backoff docs](https://dotflow-io.github.io/dotflow/nav/tutorial/task-backoff/) | [Timeout docs](https://dotflow-io.github.io/dotflow/nav/tutorial/task-timeout/)
257
304
 
@@ -274,7 +321,10 @@ def unreliable_api_call():
274
321
 
275
322
  ---
276
323
 
277
- ### Context System
324
+ </details>
325
+
326
+ <details>
327
+ <summary><strong>Context System</strong></summary>
278
328
 
279
329
  > [Context docs](https://dotflow-io.github.io/dotflow/nav/tutorial/initial-context/) | [Previous Context](https://dotflow-io.github.io/dotflow/nav/tutorial/previous-context/) | [Many Contexts](https://dotflow-io.github.io/dotflow/nav/tutorial/many-contexts/)
280
330
 
@@ -305,7 +355,10 @@ Each `Context` object contains:
305
355
 
306
356
  ---
307
357
 
308
- ### Checkpoint & Resume
358
+ </details>
359
+
360
+ <details>
361
+ <summary><strong>Checkpoint & Resume</strong></summary>
309
362
 
310
363
  > [Checkpoint docs](https://dotflow-io.github.io/dotflow/nav/tutorial/checkpoint/)
311
364
 
@@ -331,7 +384,10 @@ workflow.start(resume=True)
331
384
 
332
385
  ---
333
386
 
334
- ### Storage Providers
387
+ </details>
388
+
389
+ <details>
390
+ <summary><strong>Storage Providers</strong></summary>
335
391
 
336
392
  > [Storage docs](https://dotflow-io.github.io/dotflow/nav/tutorial/storage-default/)
337
393
 
@@ -385,32 +441,39 @@ workflow = DotFlow(config=config)
385
441
 
386
442
  ---
387
443
 
388
- ### Notifications
444
+ </details>
445
+
446
+ <details>
447
+ <summary><strong>Notifications</strong></summary>
389
448
 
390
- > [Telegram docs](https://dotflow-io.github.io/dotflow/nav/tutorial/notify-telegram/)
449
+ > [Telegram docs](https://dotflow-io.github.io/dotflow/nav/tutorial/notify-telegram/) | [Discord docs](https://dotflow-io.github.io/dotflow/nav/tutorial/notify-discord/)
391
450
 
392
- Get notified about task status changes via Telegram.
451
+ Get notified about task status changes via Telegram or Discord.
393
452
 
394
453
  ```python
395
- from dotflow import DotFlow, Config
454
+ from dotflow import Config
396
455
  from dotflow.providers import NotifyTelegram
397
- from dotflow.core.types.status import TypeStatus
398
456
 
399
- notify = NotifyTelegram(
457
+ config = Config(notify=NotifyTelegram(
400
458
  token="YOUR_BOT_TOKEN",
401
459
  chat_id=123456789,
402
- notification_type=TypeStatus.FAILED, # only notify on failures (optional)
403
- )
404
-
405
- config = Config(notify=notify)
406
- workflow = DotFlow(config=config)
460
+ ))
407
461
  ```
408
462
 
409
- Status types: `NOT_STARTED`, `IN_PROGRESS`, `COMPLETED`, `PAUSED`, `RETRY`, `FAILED`
463
+ ```python
464
+ from dotflow.providers import NotifyDiscord
465
+
466
+ config = Config(notify=NotifyDiscord(
467
+ webhook_url="https://discord.com/api/webhooks/...",
468
+ ))
469
+ ```
410
470
 
411
471
  ---
412
472
 
413
- ### Class-Based Steps
473
+ </details>
474
+
475
+ <details>
476
+ <summary><strong>Class-Based Steps</strong></summary>
414
477
 
415
478
  Return a class instance from a task, and Dotflow will automatically discover and execute all `@action`-decorated methods in source order.
416
479
 
@@ -442,7 +505,10 @@ workflow.start()
442
505
 
443
506
  ---
444
507
 
445
- ### Task Groups
508
+ </details>
509
+
510
+ <details>
511
+ <summary><strong>Task Groups</strong></summary>
446
512
 
447
513
  > [Groups docs](https://dotflow-io.github.io/dotflow/nav/tutorial/groups/)
448
514
 
@@ -459,7 +525,10 @@ workflow.start() # groups run in parallel, tasks within each group run sequenti
459
525
 
460
526
  ---
461
527
 
462
- ### Callbacks
528
+ </details>
529
+
530
+ <details>
531
+ <summary><strong>Callbacks</strong></summary>
463
532
 
464
533
  > [Task Callback docs](https://dotflow-io.github.io/dotflow/nav/tutorial/task-callback/) | [Workflow Callback docs](https://dotflow-io.github.io/dotflow/nav/tutorial/workflow-callback/)
465
534
 
@@ -486,7 +555,10 @@ workflow.start(on_success=on_success, on_failure=on_failure)
486
555
 
487
556
  ---
488
557
 
489
- ### Error Handling
558
+ </details>
559
+
560
+ <details>
561
+ <summary><strong>Error Handling</strong></summary>
490
562
 
491
563
  > [Error Handling docs](https://dotflow-io.github.io/dotflow/nav/tutorial/error-handling/) | [Keep Going docs](https://dotflow-io.github.io/dotflow/nav/tutorial/keep-going/)
492
564
 
@@ -516,7 +588,10 @@ for task in workflow.result_task():
516
588
 
517
589
  ---
518
590
 
519
- ### Async Support
591
+ </details>
592
+
593
+ <details>
594
+ <summary><strong>Async Support</strong></summary>
520
595
 
521
596
  > [Async docs](https://dotflow-io.github.io/dotflow/nav/tutorial/async-actions/)
522
597
 
@@ -539,7 +614,10 @@ workflow.start()
539
614
 
540
615
  ---
541
616
 
542
- ### Scheduler / Cron
617
+ </details>
618
+
619
+ <details>
620
+ <summary><strong>Scheduler / Cron</strong></summary>
543
621
 
544
622
  > [Cron scheduler docs](https://dotflow-io.github.io/dotflow/nav/tutorial/scheduler-cron/) | [Default scheduler](https://dotflow-io.github.io/dotflow/nav/tutorial/scheduler-default/) | [Cron overlap (concepts)](https://dotflow-io.github.io/dotflow/nav/concepts/concept-cron-overlap/)
545
623
 
@@ -588,7 +666,10 @@ The scheduler handles graceful shutdown via `SIGINT`/`SIGTERM` signals automatic
588
666
 
589
667
  ---
590
668
 
591
- ### CLI
669
+ </details>
670
+
671
+ <details>
672
+ <summary><strong>CLI</strong></summary>
592
673
 
593
674
  > [CLI docs](https://dotflow-io.github.io/dotflow/nav/how-to/cli/simple-start/)
594
675
 
@@ -630,14 +711,20 @@ Available CLI commands:
630
711
 
631
712
  | Command | Description |
632
713
  |---------|-------------|
633
- | `dotflow init` | Initialize a new Dotflow project |
714
+ | `dotflow init` | Scaffold a new project with cloud support |
634
715
  | `dotflow start` | Run a workflow |
635
716
  | `dotflow schedule` | Run a workflow on a cron schedule |
636
- | `dotflow log` | View execution logs |
717
+ | `dotflow logs` | View execution logs |
718
+ | `dotflow cloud list` | Show available cloud platforms |
719
+ | `dotflow cloud generate --platform <name>` | Generate deployment files |
720
+ | `dotflow deploy --platform <name> --project <name>` | Deploy to cloud |
637
721
 
638
722
  ---
639
723
 
640
- ### Dependency Injection via Config
724
+ </details>
725
+
726
+ <details>
727
+ <summary><strong>Dependency Injection via Config</strong></summary>
641
728
 
642
729
  The `Config` class lets you swap providers for storage, notifications, logging, and scheduling:
643
730
 
@@ -666,7 +753,10 @@ Extend Dotflow by implementing the abstract base classes:
666
753
 
667
754
  ---
668
755
 
669
- ### Results & Inspection
756
+ </details>
757
+
758
+ <details>
759
+ <summary><strong>Results & Inspection</strong></summary>
670
760
 
671
761
  After execution, inspect results directly from the workflow object:
672
762
 
@@ -697,7 +787,10 @@ workflow.task.schema() # Pydantic schema of the workflow
697
787
 
698
788
  ---
699
789
 
700
- ### Dynamic Module Import
790
+ </details>
791
+
792
+ <details>
793
+ <summary><strong>Dynamic Module Import</strong></summary>
701
794
 
702
795
  Reference tasks and callbacks by their module path string instead of importing them directly:
703
796
 
@@ -708,124 +801,12 @@ workflow.task.add(step="my_package.tasks.save_results", callback="my_package.cal
708
801
 
709
802
  ---
710
803
 
711
- ## More Examples
712
-
713
- All examples are available in the [`docs_src/`](https://github.com/dotflow-io/dotflow/tree/develop/docs_src) directory.
714
-
715
- #### Basic
716
-
717
- | Example | Description | Command |
718
- |---------|-------------|---------|
719
- | [first_steps](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/first_steps/first_steps.py) | Minimal workflow with callback | `python docs_src/first_steps/first_steps.py` |
720
- | [simple_function_workflow](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/basic/simple_function_workflow.py) | Simple function-based workflow | `python docs_src/basic/simple_function_workflow.py` |
721
- | [simple_class_workflow](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/basic/simple_class_workflow.py) | Class-based step with retry | `python docs_src/basic/simple_class_workflow.py` |
722
- | [simple_function_workflow_with_error](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/basic/simple_function_workflow_with_error.py) | Error inspection after failure | `python docs_src/basic/simple_function_workflow_with_error.py` |
723
-
724
- #### Async
725
-
726
- | Example | Description | Command |
727
- |---------|-------------|---------|
728
- | [async_action](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/async/async_action.py) | Async task functions | `python docs_src/async/async_action.py` |
729
-
730
- #### Context
731
-
732
- | Example | Description | Command |
733
- |---------|-------------|---------|
734
- | [context](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/context/context.py) | Creating and inspecting a Context | `python docs_src/context/context.py` |
735
- | [initial_context](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/initial_context/initial_context.py) | Passing initial context per task | `python docs_src/initial_context/initial_context.py` |
736
- | [previous_context](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/previous_context/previous_context.py) | Chaining context between tasks | `python docs_src/previous_context/previous_context.py` |
737
- | [many_contexts](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/context/many_contexts.py) | Using both initial and previous context | `python docs_src/context/many_contexts.py` |
738
-
739
- #### Process Modes
740
-
741
- | Example | Description | Command |
742
- |---------|-------------|---------|
743
- | [sequential](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/process_mode/sequential.py) | Sequential execution | `python docs_src/process_mode/sequential.py` |
744
- | [background](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/process_mode/background.py) | Background (non-blocking) execution | `python docs_src/process_mode/background.py` |
745
- | [parallel](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/process_mode/parallel.py) | Parallel execution | `python docs_src/process_mode/parallel.py` |
746
- | [parallel_group](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/process_mode/parallel_group.py) | Parallel groups execution | `python docs_src/process_mode/parallel_group.py` |
747
- | [sequential_group_mode](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/workflow/sequential_group_mode.py) | Sequential with named groups | `python docs_src/workflow/sequential_group_mode.py` |
748
-
749
- #### Resilience (Retry, Backoff, Timeout)
750
-
751
- | Example | Description | Command |
752
- |---------|-------------|---------|
753
- | [retry](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/retry/retry.py) | Retry on function and class steps | `python docs_src/retry/retry.py` |
754
- | [retry_delay](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/retry/retry_delay.py) | Retry with delay between attempts | `python docs_src/retry/retry_delay.py` |
755
- | [backoff](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/backoff/backoff.py) | Exponential backoff on retries | `python docs_src/backoff/backoff.py` |
756
- | [timeout](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/timeout/timeout.py) | Timeout per task execution | `python docs_src/timeout/timeout.py` |
757
-
758
- #### Callbacks
759
-
760
- | Example | Description | Command |
761
- |---------|-------------|---------|
762
- | [task_callback](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/callback/task_callback.py) | Per-task callback on completion | `python docs_src/callback/task_callback.py` |
763
- | [workflow_callback_success](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/callback/workflow_callback_success.py) | Workflow-level success callback | `python docs_src/callback/workflow_callback_success.py` |
764
- | [workflow_callback_failure](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/callback/workflow_callback_failure.py) | Workflow-level failure callback | `python docs_src/callback/workflow_callback_failure.py` |
765
-
766
- #### Error Handling
767
-
768
- | Example | Description | Command |
769
- |---------|-------------|---------|
770
- | [errors](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/errors/errors.py) | Inspecting task errors and retry count | `python docs_src/errors/errors.py` |
771
- | [keep_going_true](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/workflow/keep_going_true.py) | Continue workflow after task failure | `python docs_src/workflow/keep_going_true.py` |
772
-
773
- #### Groups
774
-
775
- | Example | Description | Command |
776
- |---------|-------------|---------|
777
- | [step_with_groups](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/group/step_with_groups.py) | Tasks in named parallel groups | `python docs_src/group/step_with_groups.py` |
778
-
779
- #### Storage
780
-
781
- | Example | Description | Command |
782
- |---------|-------------|---------|
783
- | [storage_file](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/storage/storage_file.py) | File-based JSON storage | `python docs_src/storage/storage_file.py` |
784
- | [storage_s3](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/storage/storage_s3.py) | AWS S3 storage | `python docs_src/storage/storage_s3.py` |
785
- | [storage_gcs](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/storage/storage_gcs.py) | Google Cloud Storage | `python docs_src/storage/storage_gcs.py` |
786
-
787
- #### Checkpoint & Resume
788
-
789
- | Example | Description | Command |
790
- |---------|-------------|---------|
791
- | [checkpoint](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/checkpoint/checkpoint.py) | Resume workflow from last checkpoint | `python docs_src/checkpoint/checkpoint.py` |
792
-
793
- #### Notifications
794
-
795
- | Example | Description | Command |
796
- |---------|-------------|---------|
797
- | [notify_telegram](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/notify/notify_telegram.py) | Telegram notifications on failure | `python docs_src/notify/notify_telegram.py` |
798
-
799
- #### Config & Providers
804
+ </details>
800
805
 
801
- | Example | Description | Command |
802
- |---------|-------------|---------|
803
- | [config](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/config/config.py) | Full Config with storage, notify, log | `python docs_src/config/config.py` |
804
- | [storage_provider](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/config/storage_provider.py) | Swapping storage providers | `python docs_src/config/storage_provider.py` |
805
- | [notify_provider](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/config/notify_provider.py) | Swapping notification providers | `python docs_src/config/notify_provider.py` |
806
- | [log_provider](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/config/log_provider.py) | Custom log provider | `python docs_src/config/log_provider.py` |
807
806
 
808
- #### Results & Output
807
+ ## More Examples
809
808
 
810
- | Example | Description | Command |
811
- |---------|-------------|---------|
812
- | [step_function_result_task](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/output/step_function_result_task.py) | Inspect task results (function) | `python docs_src/output/step_function_result_task.py` |
813
- | [step_function_result_context](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/output/step_function_result_context.py) | Inspect context results (function) | `python docs_src/output/step_function_result_context.py` |
814
- | [step_function_result_storage](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/output/step_function_result_storage.py) | Inspect storage results (function) | `python docs_src/output/step_function_result_storage.py` |
815
- | [step_class_result_task](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/output/step_class_result_task.py) | Inspect task results (class) | `python docs_src/output/step_class_result_task.py` |
816
- | [step_class_result_context](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/output/step_class_result_context.py) | Inspect context results (class) | `python docs_src/output/step_class_result_context.py` |
817
- | [step_class_result_storage](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/output/step_class_result_storage.py) | Inspect storage results (class) | `python docs_src/output/step_class_result_storage.py` |
818
-
819
- #### CLI
820
-
821
- | Example | Description | Command |
822
- |---------|-------------|---------|
823
- | [simple_cli](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/basic/simple_cli.py) | Basic CLI execution | `dotflow start --step docs_src.basic.simple_cli.simple_step` |
824
- | [cli_with_callback](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/cli/cli_with_callback.py) | CLI with callback function | `dotflow start --step docs_src.cli.cli_with_callback.simple_step --callback docs_src.cli.cli_with_callback.callback` |
825
- | [cli_with_initial_context](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/cli/cli_with_initial_context.py) | CLI with initial context | `dotflow start --step docs_src.cli.cli_with_initial_context.simple_step --initial-context abc` |
826
- | [cli_with_mode](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/cli/cli_with_mode.py) | CLI with execution mode | `dotflow start --step docs_src.cli.cli_with_mode.simple_step --mode sequential` |
827
- | [cli_with_output_context](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/cli/cli_with_output_context.py) | CLI with file storage output | `dotflow start --step docs_src.cli.cli_with_output_context.simple_step --storage file` |
828
- | [cli_with_path](https://github.com/dotflow-io/dotflow/blob/develop/docs_src/cli/cli_with_path.py) | CLI with custom storage path | `dotflow start --step docs_src.cli.cli_with_path.simple_step --path .storage --storage file` |
809
+ All examples are available in the [`docs_src/`](https://github.com/dotflow-io/dotflow/tree/develop/docs_src) directory.
829
810
 
830
811
  ## Commit Style
831
812