queue-max 0.1.0__py3-none-any.whl

queue_max-0.1.0.dist-info/METADATA

@@ -0,0 +1,233 @@
+Metadata-Version: 2.4
+Name: queue-max
+Version: 0.1.0
+Summary: Task queue with SQLite sharding, rate limiting, and circuit breaker
+Author: Alexandre All
+License: MIT
+Project-URL: homepage, https://github.com/all451/queue-max
+Project-URL: repository, https://github.com/all451/queue-max
+Keywords: queue,task-queue,sqlite,background-tasks,worker,sharding
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: System :: Distributed Computing
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: typing-extensions>=4.5.0
+Provides-Extra: webhook
+Requires-Dist: requests>=2.31.0; extra == "webhook"
+Provides-Extra: django
+Requires-Dist: Django>=3.2; extra == "django"
+Provides-Extra: fastapi
+Requires-Dist: fastapi>=0.100.0; extra == "fastapi"
+Provides-Extra: flask
+Requires-Dist: Flask>=2.0; extra == "flask"
+Provides-Extra: all
+Requires-Dist: requests>=2.31.0; extra == "all"
+Requires-Dist: Django>=3.2; extra == "all"
+Requires-Dist: fastapi>=0.100.0; extra == "all"
+Requires-Dist: Flask>=2.0; extra == "all"
+Dynamic: license-file
+
+# Queue Max
+
+Task queue library with SQLite persistence, sharding, rate limiting, and circuit breaker.
+
+No Redis or RabbitMQ required. Zero external dependencies (except typing-extensions).
+
+## Installation
+
+```bash
+pip install queue-max
+```
+
+With framework integrations:
+
+```bash
+pip install queue-max[django]
+pip install queue-max[fastapi]
+pip install queue-max[flask]
+```
+
+## Quick Start
+
+```python
+from queue_max import Queue, Worker
+
+# Create queue
+queue = Queue()
+
+# Enqueue a job
+queue.enqueue({"task": "send_email", "to": "user@example.com"}, priority=2)
+
+# Define processor
+def process(payload):
+    print(f"Processing: {payload}")
+
+# Start worker
+worker = Worker("worker-1", process, queue)
+worker.start()
+```
+
+## Features
+
+**SQLite Persistence** -- Jobs are stored in SQLite with WAL mode. No external services needed.
+
+**Physical Sharding** -- Multiple .db files allow concurrent read/write across workers.
+
+**Rate Limiting** -- Token bucket algorithm shared across all workers. Configurable per minute.
+
+**Circuit Breaker** -- Stops calling failing services after N consecutive errors. Recovers automatically.
+
+**Retry with Backoff** -- Exponential backoff with jitter for failed jobs. Configurable max retries.
+
+**Heartbeat and Recovery** -- Workers send periodic heartbeats. Orphaned jobs are recovered automatically.
+
+**Priority Queues** -- Three levels: low (0), medium (1), high (2).
+
+**CLI** -- Built-in command line for stats, workers, and queue management.
+
+## Configuration via Environment Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| NUM_SHARDS | 6 | Number of shard databases |
+| RATE_LIMIT_MAX | 160 | Max requests per minute |
+| QUEUE_MAX_RETRIES | 3 | Default max retry attempts |
+| DB_BUSY_TIMEOUT | 30000 | SQLite busy timeout (ms) |
+| HEARTBEAT_INTERVAL | 5000 | Worker heartbeat interval (ms) |
+| STUCK_TIMEOUT | 30000 | Orphan job timeout (ms) |
+| DATA_DIR | ./data | Directory for shard files |
+
+## API Overview
+
+### Queue
+
+```python
+from queue_max import Queue
+
+queue = Queue(shards=6, rate_limit=160, max_retries=3)
+
+# Enqueue jobs
+queue.enqueue(payload, pagina_id=None, priority=0, max_retries=None)
+queue.enqueue_batch([{"payload": {...}}, ...])
+
+# Process jobs
+job = queue.pop_job(worker_id)
+queue.complete_job(job_id, shard_id)
+queue.fail_job(job_id, shard_id, error, permanent=False)
+
+# Management
+queue.retry_failed_jobs()
+queue.cleanup_old_jobs(days=7)
+queue.recover_orphans()
+stats = queue.get_stats()
+```
+
+### Worker
+
+```python
+from queue_max import Worker, WorkerPool
+
+worker = Worker("worker-1", process_function, queue)
+worker.start()
+worker.stop()
+
+pool = WorkerPool([worker1, worker2])
+pool.start_all()
+pool.stop_all()
+```
+
+### Decorator
+
+```python
+from queue_max import task
+
+@task(priority=2, max_retries=3)
+def send_email(to: str, subject: str):
+    return send(to, subject)
+
+send_email.delay("user@example.com", "Hello")
+```
+
+### CLI
+
+```bash
+queue-max stats
+queue-max worker --function mymodule:myfunction --workers 4
+queue-max enqueue --payload '{"task": "test"}' --priority 2
+queue-max list --status failed --limit 20
+queue-max retry
+queue-max purge --days 7
+```
+
+## Framework Integrations
+
+### Django
+
+```python
+# settings.py
+INSTALLED_APPS = ["queue_max.contrib.django", ...]
+QUEUE_MAX = {"SHARDS": 4, "RATE_LIMIT": 160}
+
+# tasks.py
+from queue_max.contrib.django import task
+@task
+def my_task(user_id): ...
+```
+
+Management commands: `python manage.py queue_worker`, `queue_stats`, `queue_purge`.
+
+### FastAPI
+
+```python
+from fastapi import FastAPI
+from queue_max.contrib.fastapi import QueueMiddleware
+
+app = FastAPI()
+app.add_middleware(QueueMiddleware, max_workers=4)
+```
+
+### Flask
+
+```python
+from flask import Flask
+from queue_max.contrib.flask import QueueExtension
+
+app = Flask(__name__)
+queue = QueueExtension(app)
+
+@queue.task
+def my_task(): ...
+```
+
+## Performance
+
+| Cenário | Throughput |
+|---------|-----------|
+| Burst (20 workers, 10 shards) | **~3.300 jobs/sec** |
+| Contenção (10 workers, 1 shard) | **~1.660 jobs/sec** |
+| Com 30% de falhas (8 workers) | **Estável** — circuit breaker não trip |
+
+- Max queue size: 1M+ jobs per shard
+- [Resultados detalhados dos stress tests](docs/stress-test.md)
+
+## Running Tests
+
+```bash
+pip install -e ".[dev]"
+pytest tests/ -v
+```
+
+## License
+
+MIT

queue_max-0.1.0.dist-info/RECORD

@@ -0,0 +1,30 @@
+queue_max/__init__.py,sha256=pVW2jQZZR9FDcgGCQH9idcHb67T13QlwO3u4mBnzTW0,1679
+queue_max/cli.py,sha256=ozBGmcqg2PT0LJ__0xonkhvQd1CR4bVCkc58_RGyzlE,11214
+queue_max/exceptions.py,sha256=M9HTmNwkPQgBeQOuj8cUfUdppqzFcGfKBRZMJXyDAHE,656
+queue_max/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+queue_max/contrib/__init__.py,sha256=p4u__-xXmLjszOqJh0wTvvFK6rnzUpz65n4xgfnWOd0,189
+queue_max/contrib/django/__init__.py,sha256=jMKt6ZOEkSJzqXKGKIaINk1IhDuRjBi-W0idzf4L3zY,1508
+queue_max/contrib/django/management/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+queue_max/contrib/django/management/commands/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+queue_max/contrib/django/management/commands/queue_purge.py,sha256=5JLfMh4qs_tkdmmN2vYZuWVQZdfr7c43T6seQ1wLrCw,634
+queue_max/contrib/django/management/commands/queue_stats.py,sha256=JSBXPqinpwsmWWF68oWG9N0pbUEAPc-O57S3WGqtIJw,1376
+queue_max/contrib/django/management/commands/queue_worker.py,sha256=0Sf1bvoKz9pPGxqkfukc19ws4PyKBaRIWu2LZdBBbxU,1975
+queue_max/contrib/fastapi/__init__.py,sha256=iKA1YcnJsfqypG_vbJn1AoNhkU9FLhjfZQJlmQktTcw,3361
+queue_max/contrib/flask/__init__.py,sha256=_2vudihmqns6U3soeJYw83Rn5xTjJmPlEf72CrWz4f0,2654
+queue_max/core/__init__.py,sha256=EraVVCNdXC0GjJJqUxWM_0O7cvRAOYusjYoOxoqBDaE,424
+queue_max/core/circuit_breaker.py,sha256=Afd-KGgXG9WItXVf2wMKZ8Ejmh7SNDZ37yzeCxqods8,5581
+queue_max/core/database.py,sha256=SPIYkbKNTOdGSCoB4vwfp5ZglJ8TG7MSag2Tvk-eGYU,13690
+queue_max/core/decorator.py,sha256=xevadJn6MOJU3QGQv9WLjwIlAjhIAxVGqTwyJyupl00,11799
+queue_max/core/queue.py,sha256=OCuPzwdywGg5qske2Nk-Ke7Ia0DBuBR8AU7VbBqctb4,14718
+queue_max/core/rate_limiter.py,sha256=atifI2hciW-v6ymytZ1PK9wzG0C6G0MFvRrNeL1MaKs,7503
+queue_max/core/worker.py,sha256=ft06MwSfLKmqpEzZqp3hLy7bMB3CJxASmqZa7bEwXyY,16328
+queue_max/models/__init__.py,sha256=mscNDISJQODwSVSMEehwzcmEzFplIZZRUuRfvxFq-Os,170
+queue_max/models/job.py,sha256=Jv6HvjqThowCRHyXEM-5aGJnx_GBO1A4pRLdOBUFueI,12151
+queue_max/utils/__init__.py,sha256=sTK3MKr466GUciFjaNgvUQuqorDRhpTsTQdl4ORiwYM,421
+queue_max/utils/helpers.py,sha256=KYUPMhZEmv2qAsxO9uiR2Y4zLpsQVNvbd8PBq2og17E,4531
+queue_max-0.1.0.dist-info/licenses/LICENSE,sha256=n-RhQRnO7EIKDJj0Wfg1C_M8zFWTlgLZG3X1JEBVmnY,1083
+queue_max-0.1.0.dist-info/METADATA,sha256=wCJoiTMzyx0GtefEoaRMmRR6sDqX7JfIxjNLvoJCz1I,5913
+queue_max-0.1.0.dist-info/WHEEL,sha256=aeYiig01lYGDzBgS8HxWXOg3uV61G9ijOsup-k9o1sk,91
+queue_max-0.1.0.dist-info/entry_points.txt,sha256=fxGeS2pxnBTZQmHptITP9OSQXB5MBmHe8BuaL_CEbJk,49
+queue_max-0.1.0.dist-info/top_level.txt,sha256=C3fzMf7hh4gllaPe6_UvcxHph7xbamwBhHfxADmwgno,10
+queue_max-0.1.0.dist-info/RECORD,,

queue_max-0.1.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (82.0.1)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

queue_max-0.1.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+queue-max = queue_max.cli:main

queue_max-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Robusta Queue Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

queue_max-0.1.0.dist-info/top_level.txt

@@ -0,0 +1 @@
+queue_max