pytaskq 0.3.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,262 @@
1
+ Metadata-Version: 2.4
2
+ Name: pytaskq
3
+ Version: 0.3.0
4
+ Summary: Background jobs for Python — simple to start, safe in production. Async-first task queue with a live dashboard, built on Redis.
5
+ Author: Vraj Patel
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/vrajpatelm/PyTaskQ
8
+ Project-URL: Documentation, https://github.com/vrajpatelm/PyTaskQ#readme
9
+ Project-URL: Issues, https://github.com/vrajpatelm/PyTaskQ/issues
10
+ Project-URL: Changelog, https://github.com/vrajpatelm/PyTaskQ/blob/main/CHANGELOG.md
11
+ Keywords: task queue,background jobs,asyncio,redis,fastapi,worker,cron,celery alternative
12
+ Classifier: Development Status :: 4 - Beta
13
+ Classifier: Framework :: AsyncIO
14
+ Classifier: Framework :: FastAPI
15
+ Classifier: Intended Audience :: Developers
16
+ Classifier: Operating System :: OS Independent
17
+ Classifier: Programming Language :: Python :: 3
18
+ Classifier: Programming Language :: Python :: 3 :: Only
19
+ Classifier: Programming Language :: Python :: 3.10
20
+ Classifier: Programming Language :: Python :: 3.11
21
+ Classifier: Programming Language :: Python :: 3.12
22
+ Classifier: Programming Language :: Python :: 3.13
23
+ Classifier: Topic :: System :: Distributed Computing
24
+ Classifier: Topic :: Software Development :: Libraries :: Python Modules
25
+ Requires-Python: >=3.10
26
+ Description-Content-Type: text/markdown
27
+ License-File: LICENSE
28
+ Requires-Dist: fastapi>=0.100.0
29
+ Requires-Dist: uvicorn[standard]>=0.23.0
30
+ Requires-Dist: redis>=5.0.0
31
+ Requires-Dist: pydantic>=2.0.0
32
+ Requires-Dist: pydantic-settings>=2.0.0
33
+ Requires-Dist: croniter>=2.0.0
34
+ Requires-Dist: python-dotenv>=1.0.0
35
+ Requires-Dist: opentelemetry-api>=1.20
36
+ Provides-Extra: tracing
37
+ Requires-Dist: opentelemetry-sdk>=1.20; extra == "tracing"
38
+ Requires-Dist: opentelemetry-exporter-otlp>=1.20; extra == "tracing"
39
+ Provides-Extra: demo
40
+ Requires-Dist: numpy>=1.26.0; extra == "demo"
41
+ Requires-Dist: Pillow>=10.0.1; extra == "demo"
42
+ Provides-Extra: dev
43
+ Requires-Dist: pytest>=8.0.0; extra == "dev"
44
+ Requires-Dist: pytest-asyncio>=0.23.0; extra == "dev"
45
+ Requires-Dist: fakeredis[lua]>=2.20.0; extra == "dev"
46
+ Requires-Dist: httpx>=0.26.0; extra == "dev"
47
+ Requires-Dist: ruff; extra == "dev"
48
+ Requires-Dist: build; extra == "dev"
49
+ Requires-Dist: twine; extra == "dev"
50
+ Dynamic: license-file
51
+
52
+ <div align="center">
53
+
54
+ <img src="https://raw.githubusercontent.com/vrajpatelm/PyTaskQ/main/pytaskq/static/favicon.svg" width="72" alt="PyTaskQ logo" />
55
+
56
+ # PyTaskQ
57
+
58
+ **Background jobs for Python — simple to start, safe in production.**
59
+
60
+ Decorate a function, call `.delay()`, and watch it run in a live dashboard.<br/>
61
+ Async-first, self-hosted, built on FastAPI and Redis.
62
+
63
+ [![PyPI](https://img.shields.io/pypi/v/pytaskq?color=6366f1)](https://pypi.org/project/pytaskq/)
64
+ [![Python](https://img.shields.io/badge/python-3.10%2B-3776AB?logo=python&logoColor=white)](https://pypi.org/project/pytaskq/)
65
+ [![CI](https://github.com/vrajpatelm/PyTaskQ/actions/workflows/ci.yml/badge.svg)](https://github.com/vrajpatelm/PyTaskQ/actions/workflows/ci.yml)
66
+ [![License](https://img.shields.io/badge/license-MIT-6366f1)](https://github.com/vrajpatelm/PyTaskQ/blob/main/LICENSE)
67
+
68
+ [Quick start](#quick-start) · [Dashboard](#the-dashboard) · [Accounts](#accounts) · [Configuration](#configuration) · [How it works](#how-it-works) · [API](#rest-api) · [Docker](#docker)
69
+
70
+ <br/>
71
+
72
+ <img src="https://raw.githubusercontent.com/vrajpatelm/PyTaskQ/main/docs/images/hero.png" alt="The PyTaskQ dashboard in light and dark themes" width="100%" />
73
+
74
+ </div>
75
+
76
+ ## Why PyTaskQ
77
+
78
+ - **Plain Python.** One decorator. Sync, `async def`, CPU-heavy and I/O-bound tasks each run in the right pool automatically.
79
+ - **See everything.** A live dashboard shows every job's arguments, timing, result and error, and lets you run, cancel and replay jobs.
80
+ - **Built for failure.** Retries with backoff, a dead-letter queue, crash recovery, fencing tokens and graceful shutdown come built in.
81
+ - **Self-hosted.** Accounts, roles and invite links are included. Redis is the only thing you need to run.
82
+
83
+ ## Quick start
84
+
85
+ You need Python 3.10+ and Redis (`docker run -p 6379:6379 redis:7` is enough).
86
+
87
+ ```bash
88
+ pip install pytaskq
89
+ ```
90
+
91
+ **1. Write tasks**
92
+
93
+ ```python
94
+ # tasks.py
95
+ from pytaskq import PyTaskQ
96
+
97
+ queue = PyTaskQ()
98
+
99
+ @queue.task(type="io", priority="high", max_retries=5)
100
+ def send_order_confirmation(order_id: int, email: str):
101
+ """Email the order confirmation and receipt."""
102
+ ...
103
+
104
+ @queue.task # async def is detected automatically
105
+ async def sync_inventory(sku: str):
106
+ ...
107
+
108
+ @queue.task(cron="0 2 * * *") # every night at 02:00, once across all workers
109
+ def nightly_sales_report():
110
+ ...
111
+ ```
112
+
113
+ **2. Start a worker and the dashboard**
114
+
115
+ ```bash
116
+ pytaskq worker --app tasks:queue
117
+ pytaskq dashboard --app tasks:queue # → http://localhost:8000
118
+ ```
119
+
120
+ On first start, the dashboard prints a **setup code** in its log. Open the dashboard, enter the code and create your admin account.
121
+
122
+ **3. Queue jobs**
123
+
124
+ ```python
125
+ await send_order_confirmation.delay(10481, email="ada@example.com") # from async code
126
+ send_order_confirmation.delay_sync(10482, email="lin@example.com") # from sync code
127
+
128
+ result = await queue.wait(task_id, timeout=30) # {"status": "Success", "result": ..., "duration_ms": ...}
129
+ ```
130
+
131
+ You can also queue jobs from the dashboard or over HTTP. Want to try it without writing code? Run `pytaskq worker --app pytaskq.task_registery:queue` and use the built-in demo tasks (`pip install "pytaskq[demo]"`).
132
+
133
+ ## The dashboard
134
+
135
+ Live updates, light and dark themes, and a layout that works on phones.
136
+
137
+ **Jobs.** The lifecycle strip shows where every job is: scheduled, queued, running, succeeded or dead. Below it, filter recent jobs by status or task, or search by ID.
138
+
139
+ <img src="https://raw.githubusercontent.com/vrajpatelm/PyTaskQ/main/docs/images/dashboard-dark.png" alt="Jobs view with the lifecycle strip and a live job list" width="100%" />
140
+
141
+ **Job details.** Timings, the worker that ran it, attempts, arguments, and the result or error. Waiting jobs can be cancelled.
142
+
143
+ <img src="https://raw.githubusercontent.com/vrajpatelm/PyTaskQ/main/docs/images/job-details.png" alt="Job detail panel with timings, arguments and result" width="100%" />
144
+
145
+ **Run a task.** Each task's form is built from its function signature: required fields, types, defaults, priority and delay.
146
+
147
+ <img src="https://raw.githubusercontent.com/vrajpatelm/PyTaskQ/main/docs/images/run-task.png" alt="Run a task form generated from a function signature" width="100%" />
148
+
149
+ **Dead letters.** Jobs that failed every retry, with their last error. Replay them once the cause is fixed.
150
+
151
+ <img src="https://raw.githubusercontent.com/vrajpatelm/PyTaskQ/main/docs/images/dead-letters.png" alt="Dead letters with their errors and replay buttons" width="100%" />
152
+
153
+ **Workers.** Host, load, jobs done and failed, pool sizes, uptime and heartbeat health.
154
+
155
+ <img src="https://raw.githubusercontent.com/vrajpatelm/PyTaskQ/main/docs/images/workers.png" alt="Worker fleet with health and throughput" width="100%" />
156
+
157
+ ## Accounts
158
+
159
+ PyTaskQ has its own accounts, so you can expose it safely without an external service.
160
+
161
+ <table>
162
+ <tr>
163
+ <td width="60%" valign="middle">
164
+
165
+ **First start.** A one-time setup code is printed in the server log. Only someone with access to the server can create the first admin.
166
+
167
+ Lost it? Run `pytaskq users setup-code`. For automated installs, set `PYTASKQ_ADMIN_USER` and `PYTASKQ_ADMIN_PASSWORD` instead.
168
+
169
+ </td>
170
+ <td width="40%" align="center">
171
+ <img src="https://raw.githubusercontent.com/vrajpatelm/PyTaskQ/main/docs/images/setup.png" alt="First-run setup screen" width="300" />
172
+ </td>
173
+ </tr>
174
+ </table>
175
+
176
+ **Your team.** Admins send single-use invite links (valid 7 days), change roles and disable accounts from *Settings → Team*.
177
+
178
+ <img src="https://raw.githubusercontent.com/vrajpatelm/PyTaskQ/main/docs/images/team.png" alt="Team settings with roles and an invite link" width="100%" />
179
+
180
+ | Role | Can |
181
+ |---|---|
182
+ | **Viewer** | See jobs, workers and dead letters |
183
+ | **Operator** | Also run, cancel, replay and delete jobs |
184
+ | **Admin** | Also invite people, change roles and disable accounts |
185
+
186
+ - **Scripts** use a personal API token from *Settings → API tokens*: `Authorization: Bearer ptq_…`. The token has its owner's role.
187
+ - **Locked out?** Run `pytaskq users reset-password NAME` on the server. `pytaskq users list | create | set-role | delete | setup-code` cover the rest.
188
+ - **Security.** Passwords are hashed with scrypt. Only SHA-256 hashes of sessions, invites and tokens are stored. Cookies are `HttpOnly` and `SameSite`, cross-site writes are blocked, and failed sign-ins are throttled. Serve the dashboard over HTTPS in production.
189
+ - **Local development:** `PYTASKQ_AUTH=off` turns sign-in off. Never use it on a server others can reach.
190
+
191
+ ## Configuration
192
+
193
+ | Task option | Default | What it does |
194
+ |---|---|---|
195
+ | `type` | `"async"` for `async def`, else `"cpu"` | `"cpu"` process pool · `"io"` thread pool · `"async"` event loop |
196
+ | `priority` | `"default"` | `"high"`, `"default"` or `"low"` queue |
197
+ | `max_retries` | `3` | Retries with jittered backoff before dead-lettering |
198
+ | `timeout` | `300` s | Abandon and retry an attempt after this long |
199
+ | `cron` | — | 5-field cron schedule |
200
+ | `name` | function name | Registered task name |
201
+
202
+ App-wide defaults: `PyTaskQ(default_priority="low", max_retries=5, timeout=60)`.
203
+
204
+ Every server setting is an environment variable or a line in `.env`, and all of them are documented in [`.env.example`](https://github.com/vrajpatelm/PyTaskQ/blob/main/.env.example). The ones you'll most likely change are `REDIS_URL`, `MAX_RETRIES`, `TASK_TIMEOUT` and `QUEUE_CAPACITY`.
205
+
206
+ | Command | Purpose |
207
+ |---|---|
208
+ | `pytaskq worker --app m:queue [--cpu-workers N --io-workers N]` | Run a worker |
209
+ | `pytaskq dashboard [--app m:queue] [--port 8000]` | Dashboard and REST API |
210
+ | `pytaskq tasks --app m:queue` | List tasks and their settings |
211
+ | `pytaskq users …` | Manage accounts from the server |
212
+ | `pytaskq ping` | Check Redis and count workers |
213
+
214
+ ## How it works
215
+
216
+ <img src="https://raw.githubusercontent.com/vrajpatelm/PyTaskQ/main/docs/images/architecture.svg" alt="PyTaskQ architecture" width="100%" />
217
+
218
+ 1. **Reliable pop.** A worker atomically moves a job into its own processing list, so a crash never loses it.
219
+ 2. **Crash recovery.** Workers send a heartbeat every 10 s. An elected leader re-queues work from any worker that has been silent for 30 s.
220
+ 3. **Fencing tokens.** A Lua script only accepts results carrying the current token, so a late result from a hung worker can't overwrite a newer one.
221
+ 4. **Retries, then dead letters.** Failures back off with jitter. After `max_retries`, the job is dead-lettered with its last error.
222
+ 5. **Graceful shutdown.** Ctrl+C finishes running jobs before exiting.
223
+ 6. **Safe outbound calls.** Webhooks (`on_success_url=`) are signed with HMAC and retried. Use `pytaskq.net.assert_safe_url` to block requests to private networks.
224
+
225
+ ## REST API
226
+
227
+ Interactive docs are served at `/docs`, and the full reference is in [docs/API.md](https://github.com/vrajpatelm/PyTaskQ/blob/main/docs/API.md). Every route except `/health` and sign-in needs a session or API token.
228
+
229
+ | Method | Route | |
230
+ |---|---|---|
231
+ | `POST` | `/task/enqueue` | Queue a job (`task_name`, `args`, `kwargs`, `priority`, `delay_seconds`) |
232
+ | `GET` | `/task/{id}` | Status, timings, result or error |
233
+ | `POST` | `/task/{id}/cancel` | Cancel a waiting job |
234
+ | `GET` | `/tasks` | Recent jobs, filterable by status and task |
235
+ | `GET` | `/metrics`, `/system/workers`, `/dlq` | Queue, worker and dead-letter state |
236
+ | `POST` | `/dlq/replay/{id}`, `/dlq/retry_all` | Replay dead letters |
237
+
238
+ ## Docker
239
+
240
+ ```bash
241
+ docker compose up --build # Redis + dashboard + worker + Jaeger
242
+ docker compose logs web | grep -A4 "setup code"
243
+ ```
244
+
245
+ Point `PYTASKQ_APP` and the worker `command:` in `docker-compose.yml` at your own module. Accounts and jobs persist in the `redis-data` volume.
246
+
247
+ ## Development
248
+
249
+ ```bash
250
+ pip install -e ".[dev,demo]"
251
+ python -m pytest # in-memory via fakeredis, no Redis needed
252
+
253
+ cd frontend && npm install
254
+ npm run dev # dashboard with hot reload
255
+ npm run build # writes the bundle into pytaskq/static/
256
+ ```
257
+
258
+ See [CONTRIBUTING](https://github.com/vrajpatelm/PyTaskQ/blob/main/docs/CONTRIBUTING.md), [RELEASING](https://github.com/vrajpatelm/PyTaskQ/blob/main/docs/RELEASING.md) and the [CHANGELOG](https://github.com/vrajpatelm/PyTaskQ/blob/main/CHANGELOG.md).
259
+
260
+ ## License
261
+
262
+ MIT
@@ -0,0 +1,34 @@
1
+ pytaskq/__init__.py,sha256=0MiA8HP8lgsW2ckk9Di4ZbTDTcLAY-4C-A15l7bNS24,168
2
+ pytaskq/__main__.py,sha256=bn4M2RWgqTq_eiiLwO3sUdP2pWtGj90wbsy6dUzb9nU,68
3
+ pytaskq/app.py,sha256=FrDKbm-P4f0-wAZjK-jh1SwXvIPvHlrajYWGefaKzek,13348
4
+ pytaskq/cli.py,sha256=eiFtGL9qt7462Hisa_5cN1BPVCJ2v8iy4_j6XBSbfRA,9631
5
+ pytaskq/main.py,sha256=WWAt85uxRkR0rdNbgtqGf0CXOqElSGiXHG0TVtWCnKA,4333
6
+ pytaskq/net.py,sha256=74TBzEPKQRd0FtnreauOcTS0zxLwsc3HYV4rEA-PqlE,2415
7
+ pytaskq/task_registery.py,sha256=KjvTOQ8KNTc67Zw1TaqrtzHn7KtxOPfYGjPYdyGnqHo,4983
8
+ pytaskq/tracing.py,sha256=YCtmZWem83CldAe71PdhPip1_N8rbkCoCehw3Pb7fvI,3997
9
+ pytaskq/worker.py,sha256=QswwpFbosUKowQIYtSU4T_Yce-lK0LvdwJCN09NtNNE,25086
10
+ pytaskq/api/router.py,sha256=upAy9-wmma-fcbJ1GYrrZfLrNpEIUWnUDv6d5Hk2ubY,620
11
+ pytaskq/api/v1/auth.py,sha256=M7wFuXe6qra_w1aAkwESJX7xEmuETrDN-AQbHmnSFg0,9179
12
+ pytaskq/api/v1/dashboard.py,sha256=zxVMDcdr6gCGlHjh-HiisJ_nEVW8JPEYGqI2847X06k,3582
13
+ pytaskq/api/v1/dlq.py,sha256=BwATFsyWMPBxBbpLQbzcnMzjGoL9R9IfnzfcCk4xOII,3946
14
+ pytaskq/api/v1/system.py,sha256=rFc6260WF_zBV0lpjykgDvhjt7ac_pVOBLHny9CYG98,2280
15
+ pytaskq/api/v1/tasks.py,sha256=q3t_sbGYGnGQhi1FfunLhOlZnKk7ZGKJHxFAtny_jIA,4580
16
+ pytaskq/core/auth.py,sha256=iUE31dbPwYihhUVB00t40D12uJX-CfRmlzl6oCm8yFc,20217
17
+ pytaskq/core/config.py,sha256=ejoH5yVG6HCZLbcKTWyXZGp-_smNw5MgpXwJxU-X-h0,2746
18
+ pytaskq/core/dependencies.py,sha256=x1bi24wAgstzocX-J9s9j_EzRnl6fSqA6h8HjNT0lfI,4167
19
+ pytaskq/core/exceptions.py,sha256=bIy2KwDMCz7gJ5EmY0n0zDgx97XA2V7k-wDP7SwXc_o,453
20
+ pytaskq/core/idempotency.py,sha256=YdXIR3RlmIROeaBVc8wfhpSLXcs3ku3Qp-6ZLCKWlkM,1917
21
+ pytaskq/core/queueing.py,sha256=yEfXFBeZCqdfCLc5ENdzPIyeIty1c_7m9X-1Yw8osKE,9899
22
+ pytaskq/core/redis_client.py,sha256=Y2qQ2A6ef7NiKNVXy12TMZu173SHNMP24kxTP_M622s,568
23
+ pytaskq/schemas/requests.py,sha256=3gk45abY80vBZbEN6NgZvsG2bEb3Sncsqxh5SKeVkW4,1102
24
+ pytaskq/services/dashboard_service.py,sha256=GIIwhbIwuixZsC7RiVd1gUmCTjc0N84zvIeU2DlRG5g,1928
25
+ pytaskq/static/favicon.svg,sha256=YbyaFh3lgkgojmkFQl1xgPBiTChlAHuX12P9rBIEOmY,9522
26
+ pytaskq/static/index.html,sha256=LGq9zTL4y1x1jv0F-Gl9-JF8Gxg4tSppHzsoszepOeA,590
27
+ pytaskq/static/assets/index-Biu-vuAP.js,sha256=yYAYZ42lDtN16GzHi8NmssDH3KpEI_rS9YvVTmGyYS8,243024
28
+ pytaskq/static/assets/index-mgpfRVrS.css,sha256=AdrJCV-5CmcXidaEtuAeVaclOkuqvME7J4MdPkMFfnI,16923
29
+ pytaskq-0.3.0.dist-info/licenses/LICENSE,sha256=_acl2wY4ygMrMCjhvSrrwjqB4_cI_34CuS2ydXBMWvo,1067
30
+ pytaskq-0.3.0.dist-info/METADATA,sha256=3k_c4TlIkSDMa1ePQ0Gf8ZjP3ohfkLx7nROJ0zqsxSc,12474
31
+ pytaskq-0.3.0.dist-info/WHEEL,sha256=mOdXR9C3wMVov7cSY8PY-XCzo3SmmWCHvGQr5csJoao,91
32
+ pytaskq-0.3.0.dist-info/entry_points.txt,sha256=QsQxLWOZQ-gakuFBQMyz3KcJVVLK-gDJR4zp_mU_R3o,45
33
+ pytaskq-0.3.0.dist-info/top_level.txt,sha256=B4HwMnrwBKpuhGEfOWIG8B_OVlss9BjqxwGVboYaIPY,8
34
+ pytaskq-0.3.0.dist-info/RECORD,,
@@ -0,0 +1,5 @@
1
+ Wheel-Version: 1.0
2
+ Generator: setuptools (84.2.1)
3
+ Root-Is-Purelib: true
4
+ Tag: py3-none-any
5
+
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ pytaskq = pytaskq.cli:main
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Vraj Patel
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1 @@
1
+ pytaskq