taskiq-redis 1.0.1__tar.gz → 1.0.3__tar.gz

{taskiq_redis-1.0.1 → taskiq_redis-1.0.3}/PKG-INFO

@@ -1,22 +1,23 @@
-Metadata-Version: 2.1
+Metadata-Version: 2.3
 Name: taskiq-redis
-Version: 1.0.1
+Version: 1.0.3
 Summary: Redis integration for taskiq
-Home-page: https://github.com/taskiq-python/taskiq-redis
 Keywords: taskiq,tasks,distributed,async,redis,result_backend
 Author: taskiq-team
 Author-email: taskiq@norely.com
-Requires-Python: >=3.8.1,<4.0.0
+Requires-Python: >=3.9,<4.0
 Classifier: Programming Language :: Python
 Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3 :: Only
 Classifier: Programming Language :: Python :: 3.8
 Requires-Dist: redis (>=5,<6)
-Requires-Dist: taskiq (>=0.11.1,<1)
+Requires-Dist: taskiq (>=0.11.12,<1)
+Project-URL: Homepage, https://github.com/taskiq-python/taskiq-redis
 Project-URL: Repository, https://github.com/taskiq-python/taskiq-redis
 Description-Content-Type: text/markdown
 
@@ -43,17 +44,17 @@ Let's see the example with the redis broker and redis async result:
 # broker.py
 import asyncio
 
-from taskiq_redis import ListQueueBroker, RedisAsyncResultBackend
+from taskiq_redis import RedisAsyncResultBackend, RedisStreamBroker
 
-redis_async_result = RedisAsyncResultBackend(
+result_backend = RedisAsyncResultBackend(
     redis_url="redis://localhost:6379",
 )
 
 # Or you can use PubSubBroker if you need broadcasting
-broker = ListQueueBroker(
+# Or ListQueueBroker if you don't want acknowledges
+broker = RedisStreamBroker(
     url="redis://localhost:6379",
-    result_backend=redis_async_result,
-)
+).with_result_backend(result_backend)
 
 
 @broker.task
@@ -77,25 +78,48 @@ Launch the workers:
 Then run the main code:
 `python3 broker.py`
 
-## PubSubBroker and ListQueueBroker configuration
 
-We have two brokers with similar interfaces, but with different logic.
-The PubSubBroker uses redis' pubsub mechanism and is very powerful,
-but it executes every task on all workers, because PUBSUB broadcasts message
-to all subscribers.
+## Brokers
 
-If you want your messages to be processed only once, please use ListQueueBroker.
-It uses redis' [LPUSH](https://redis.io/commands/lpush/) and [BRPOP](https://redis.io/commands/brpop/) commands to deal with messages.
+This package contains 6 broker implementations.
+3 broker types:
+* PubSub broker
+* ListQueue broker
+* Stream broker
 
-Brokers parameters:
-* `url` - url to redis.
-* `task_id_generator` - custom task_id genertaor.
-* `result_backend` - custom result backend.
-* `queue_name` - name of the pub/sub channel in redis.
-* `max_connection_pool_size` - maximum number of connections in pool.
-* Any other keyword arguments are passed to `redis.asyncio.BlockingConnectionPool`.
-  Notably, you can use `timeout` to set custom timeout in seconds for reconnects
-  (or set it to `None` to try reconnects indefinitely).
+Each of type is implemented for each redis architecture:
+* Single node
+* Cluster
+* Sentinel
+
+Here's a small breakdown of how they differ from eachother.
+
+
+### PubSub
+
+By default on old redis versions PUBSUB was the way of making redis into a queue.
+But using PUBSUB means that all messages delivered to all subscribed consumers.
+
+> [!WARNING]
+> This broker doesn't support acknowledgements. If during message processing
+> Worker was suddenly killed the message is going to be lost.
+
+### ListQueue
+
+This broker creates a list of messages at some key. Adding new tasks will be done
+by appending them from the left side using `lpush`, and taking them from the right side using `brpop`.
+
+> [!WARNING]
+> This broker doesn't support acknowledgements. If during message processing
+> Worker was suddenly killed the message is going to be lost.
+
+### Stream
+
+Stream brokers use redis [stream type](https://redis.io/docs/latest/develop/data-types/streams/) to store and fetch messages.
+
+> [!TIP]
+> This broker **supports** acknowledgements and therefore is fine to use in cases when data durability is
+> required.
 
 ## RedisAsyncResultBackend configuration
 
@@ -107,19 +131,21 @@ RedisAsyncResultBackend parameters:
 * Any other keyword arguments are passed to `redis.asyncio.BlockingConnectionPool`.
   Notably, you can use `timeout` to set custom timeout in seconds for reconnects
   (or set it to `None` to try reconnects indefinitely).
-> IMPORTANT: **It is highly recommended to use expire time ​​in RedisAsyncResultBackend**
+
+> [!WARNING]
+> **It is highly recommended to use expire time in RedisAsyncResultBackend**
 > If you want to add expiration, either `result_ex_time` or `result_px_time` must be set.
->```python
-># First variant
->redis_async_result = RedisAsyncResultBackend(
->    redis_url="redis://localhost:6379",
->    result_ex_time=1000,
->)
+> ```python
+> # First variant
+> redis_async_result = RedisAsyncResultBackend(
+>     redis_url="redis://localhost:6379",
+>     result_ex_time=1000,
+> )
 >
-># Second variant
->redis_async_result = RedisAsyncResultBackend(
->    redis_url="redis://localhost:6379",
->    result_px_time=1000000,
->)
->```
+> # Second variant
+> redis_async_result = RedisAsyncResultBackend(
+>     redis_url="redis://localhost:6379",
+>     result_px_time=1000000,
+> )
+> ```
 

taskiq_redis-1.0.3/README.md

@@ -0,0 +1,127 @@
+# TaskIQ-Redis
+
+Taskiq-redis is a plugin for taskiq that adds a new broker and result backend based on redis.
+
+# Installation
+
+To use this project you must have installed core taskiq library:
+```bash
+pip install taskiq
+```
+This project can be installed using pip:
+```bash
+pip install taskiq-redis
+```
+
+# Usage
+
+Let's see the example with the redis broker and redis async result:
+
+```python
+# broker.py
+import asyncio
+
+from taskiq_redis import RedisAsyncResultBackend, RedisStreamBroker
+
+result_backend = RedisAsyncResultBackend(
+    redis_url="redis://localhost:6379",
+)
+
+# Or you can use PubSubBroker if you need broadcasting
+# Or ListQueueBroker if you don't want acknowledges
+broker = RedisStreamBroker(
+    url="redis://localhost:6379",
+).with_result_backend(result_backend)
+
+
+@broker.task
+async def best_task_ever() -> None:
+    """Solve all problems in the world."""
+    await asyncio.sleep(5.5)
+    print("All problems are solved!")
+
+
+async def main():
+    task = await best_task_ever.kiq()
+    print(await task.wait_result())
+
+
+if __name__ == "__main__":
+    asyncio.run(main())
+```
+
+Launch the workers:
+`taskiq worker broker:broker`
+Then run the main code:
+`python3 broker.py`
+
+
+## Brokers
+
+This package contains 6 broker implementations.
+3 broker types:
+* PubSub broker
+* ListQueue broker
+* Stream broker
+
+Each of type is implemented for each redis architecture:
+* Single node
+* Cluster
+* Sentinel
+
+Here's a small breakdown of how they differ from eachother.
+
+
+### PubSub
+
+By default on old redis versions PUBSUB was the way of making redis into a queue.
+But using PUBSUB means that all messages delivered to all subscribed consumers.
+
+> [!WARNING]
+> This broker doesn't support acknowledgements. If during message processing
+> Worker was suddenly killed the message is going to be lost.
+
+### ListQueue
+
+This broker creates a list of messages at some key. Adding new tasks will be done
+by appending them from the left side using `lpush`, and taking them from the right side using `brpop`.
+
+> [!WARNING]
+> This broker doesn't support acknowledgements. If during message processing
+> Worker was suddenly killed the message is going to be lost.
+
+### Stream
+
+Stream brokers use redis [stream type](https://redis.io/docs/latest/develop/data-types/streams/) to store and fetch messages.
+
+> [!TIP]
+> This broker **supports** acknowledgements and therefore is fine to use in cases when data durability is
+> required.
+
+## RedisAsyncResultBackend configuration
+
+RedisAsyncResultBackend parameters:
+* `redis_url` - url to redis.
+* `keep_results` - flag to not remove results from Redis after reading.
+* `result_ex_time` - expire time in seconds (by default - not specified)
+* `result_px_time` - expire time in milliseconds (by default - not specified)
+* Any other keyword arguments are passed to `redis.asyncio.BlockingConnectionPool`.
+  Notably, you can use `timeout` to set custom timeout in seconds for reconnects
+  (or set it to `None` to try reconnects indefinitely).
+
+> [!WARNING]
+> **It is highly recommended to use expire time in RedisAsyncResultBackend**
+> If you want to add expiration, either `result_ex_time` or `result_px_time` must be set.
+> ```python
+> # First variant
+> redis_async_result = RedisAsyncResultBackend(
+>     redis_url="redis://localhost:6379",
+>     result_ex_time=1000,
+> )
+>
+> # Second variant
+> redis_async_result = RedisAsyncResultBackend(
+>     redis_url="redis://localhost:6379",
+>     result_px_time=1000000,
+> )
+> ```

{taskiq_redis-1.0.1 → taskiq_redis-1.0.3}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "taskiq-redis"
-version = "1.0.1"
+version = "1.0.3"
 description = "Redis integration for taskiq"
 authors = ["taskiq-team <taskiq@norely.com>"]
 readme = "README.md"
@@ -25,22 +25,21 @@ keywords = [
 ]
 
 [tool.poetry.dependencies]
-python = "^3.8.1"
-taskiq = ">=0.11.1,<1"
+python = "^3.9"
+taskiq = ">=0.11.12,<1"
 redis = "^5"
 
 [tool.poetry.group.dev.dependencies]
-pytest = "^7.0"
+pytest = "^8"
 mypy = "^1"
-black = "^22.3.0"
-pytest-cov = "^3.0.0"
-anyio = "^3.6.1"
-pytest-env = "^0.6.2"
+black = "^25"
+pytest-cov = "^6"
+anyio = "^4"
+pytest-env = "^1"
 fakeredis = "^2"
-pre-commit = "^2.20.0"
-pytest-xdist = { version = "^2.5.0", extras = ["psutil"] }
+pre-commit = "^4"
+pytest-xdist = { version = "^3", extras = ["psutil"] }
 ruff = "^0"
-types-redis = "^4.6.0.20240425"
 
 [tool.mypy]
 strict = true
@@ -56,6 +55,7 @@ warn_return_any = false
 [[tool.mypy.overrides]]
 module = ['redis']
 ignore_missing_imports = true
+ignore_errors = true
 strict = false
 
 [build-system]
@@ -65,7 +65,7 @@ build-backend = "poetry.core.masonry.api"
 [tool.ruff]
 # List of enabled rulsets.
 # See https://docs.astral.sh/ruff/rules/ for more information.
-select = [
+lint.select = [
     "E",   # Error
     "F",   # Pyflakes
     "W",   # Pycodestyle
@@ -92,24 +92,22 @@ select = [
     "PL",  # PyLint checks
     "RUF", # Specific to Ruff checks
 ]
-ignore = [
+lint.ignore = [
     "D105",    # Missing docstring in magic method
     "D107",    # Missing docstring in __init__
     "D212",    # Multi-line docstring summary should start at the first line
     "D401",    # First line should be in imperative mood
     "D104",    # Missing docstring in public package
     "D100",    # Missing docstring in public module
-    "ANN102",  # Missing type annotation for self in method
-    "ANN101",  # Missing type annotation for argument
     "ANN401",  # typing.Any are disallowed in `**kwargs
     "PLR0913", # Too many arguments for function call
     "D106",    # Missing docstring in public nested class
 ]
 exclude = [".venv/"]
-mccabe = { max-complexity = 10 }
+lint.mccabe = { max-complexity = 10 }
 line-length = 88
 
-[tool.ruff.per-file-ignores]
+[tool.ruff.lint.per-file-ignores]
 "tests/*" = [
     "S101",   # Use of assert detected
     "S301",   # Use of pickle detected
@@ -119,12 +117,12 @@ line-length = 88
     "D101",   # Missing docstring in public class
 ]
 
-[tool.ruff.pydocstyle]
+[tool.ruff.lint.pydocstyle]
 convention = "pep257"
 ignore-decorators = ["typing.overload"]
 
-[tool.ruff.pylint]
+[tool.ruff.lint.pylint]
 allow-magic-value-types = ["int", "str", "float"]
 
-[tool.ruff.flake8-bugbear]
+[tool.ruff.lint.flake8-bugbear]
 extend-immutable-calls = ["taskiq_dependencies.Depends", "taskiq.TaskiqDepends"]

{taskiq_redis-1.0.1 → taskiq_redis-1.0.3}/taskiq_redis/__init__.py

@@ -1,14 +1,19 @@
 """Package for redis integration."""
+
 from taskiq_redis.redis_backend import (
     RedisAsyncClusterResultBackend,
     RedisAsyncResultBackend,
     RedisAsyncSentinelResultBackend,
 )
-from taskiq_redis.redis_broker import ListQueueBroker, PubSubBroker
-from taskiq_redis.redis_cluster_broker import ListQueueClusterBroker
+from taskiq_redis.redis_broker import ListQueueBroker, PubSubBroker, RedisStreamBroker
+from taskiq_redis.redis_cluster_broker import (
+    ListQueueClusterBroker,
+    RedisStreamClusterBroker,
+)
 from taskiq_redis.redis_sentinel_broker import (
     ListQueueSentinelBroker,
     PubSubSentinelBroker,
+    RedisStreamSentinelBroker,
 )
 from taskiq_redis.schedule_source import (
     RedisClusterScheduleSource,
@@ -17,15 +22,18 @@ from taskiq_redis.schedule_source import (
 )
 
 __all__ = [
-    "RedisAsyncClusterResultBackend",
-    "RedisAsyncResultBackend",
-    "RedisAsyncSentinelResultBackend",
     "ListQueueBroker",
-    "PubSubBroker",
     "ListQueueClusterBroker",
     "ListQueueSentinelBroker",
+    "PubSubBroker",
     "PubSubSentinelBroker",
-    "RedisScheduleSource",
+    "RedisAsyncClusterResultBackend",
+    "RedisAsyncResultBackend",
+    "RedisAsyncSentinelResultBackend",
     "RedisClusterScheduleSource",
+    "RedisScheduleSource",
     "RedisSentinelScheduleSource",
+    "RedisStreamBroker",
+    "RedisStreamClusterBroker",
+    "RedisStreamSentinelBroker",
 ]

{taskiq_redis-1.0.1 → taskiq_redis-1.0.3}/taskiq_redis/exceptions.py

@@ -8,10 +8,16 @@ class TaskIQRedisError(TaskiqError):
 class DuplicateExpireTimeSelectedError(ResultBackendError, TaskIQRedisError):
     """Error if two lifetimes are selected."""
 
+    __template__ = "Choose either result_ex_time or result_px_time."
+
 
 class ExpireTimeMustBeMoreThanZeroError(ResultBackendError, TaskIQRedisError):
     """Error if two lifetimes are less or equal zero."""
 
+    __template__ = (
+        "You must select one expire time param and it must be more than zero."
+    )
+
 
 class ResultIsMissingError(TaskIQRedisError, ResultGetError):
     """Error if there is no result when trying to get it."""