vs-queue 0.1.0__tar.gz → 0.1.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {vs_queue-0.1.0 → vs_queue-0.1.1}/PKG-INFO +37 -1
- {vs_queue-0.1.0 → vs_queue-0.1.1}/README.md +36 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/pyproject.toml +1 -1
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/provider/vs_rabbitmq_queue.py +4 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/provider/vs_redis_queue.py +4 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue.egg-info/PKG-INFO +37 -1
- {vs_queue-0.1.0 → vs_queue-0.1.1}/setup.cfg +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/__init__.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/base/__init__.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/base/vs_base_consumer.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/base/vs_base_queue.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/decorator/__init__.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/decorator/vs_queue_listener.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/factory/__init__.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/factory/vs_queue_factory.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/manager/__init__.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/manager/vs_queue_manager.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/provider/__init__.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/registry/__init__.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/registry/vs_queue_registry.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/retry/__init__.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/retry/vs_retry_policy.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/schema/__init__.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue/schema/vs_message.py +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue.egg-info/SOURCES.txt +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue.egg-info/dependency_links.txt +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue.egg-info/requires.txt +0 -0
- {vs_queue-0.1.0 → vs_queue-0.1.1}/vs_queue.egg-info/top_level.txt +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: vs-queue
|
|
3
|
-
Version: 0.1.
|
|
3
|
+
Version: 0.1.1
|
|
4
4
|
Summary: Pluggable async message queue library for Viveka Sutra — Redis Streams, RabbitMQ, retry, dead letter, and annotation-based consumers
|
|
5
5
|
Project-URL: Homepage, https://vivekasutra.com/
|
|
6
6
|
Project-URL: Source, https://github.com/vivekasutra/viveka-mula
|
|
@@ -560,6 +560,42 @@ await queue.publish("orchestrator:tasks", dlq_message)
|
|
|
560
560
|
|
|
561
561
|
---
|
|
562
562
|
|
|
563
|
+
## Logging
|
|
564
|
+
|
|
565
|
+
Both backends log consumer lifecycle and per-message activity on the provider logger
|
|
566
|
+
(`vs_queue.provider.vs_redis_queue` / `vs_queue.provider.vs_rabbitmq_queue`). Every line uses a
|
|
567
|
+
`Label | key=value` format.
|
|
568
|
+
|
|
569
|
+
| Level | Line | When |
|
|
570
|
+
|---|---|---|
|
|
571
|
+
| `INFO` | `Subscribed \| queue=<q> consumer=<ClassName>` | `consumer.start()` has run and the queue / consumer group is ready — the loop is now listening |
|
|
572
|
+
| `INFO` | `Message received \| queue=<q> id=<msg_id> retry_count=<n>` | Each message pulled off the queue, before `handle()` is called |
|
|
573
|
+
| `DEBUG` | `Message processed \| queue=<q> id=<msg_id>` | `handle()` returned and the message was acked |
|
|
574
|
+
| `INFO` | `Unsubscribed \| queue=<q> consumer=<ClassName>` | The subscribe loop exited (graceful shutdown or permanent failure) |
|
|
575
|
+
| `ERROR` | `Message failed, no retry policy \| id=<msg_id> error=<e>` | `handle()` raised and no `VsRetryPolicy` was supplied |
|
|
576
|
+
|
|
577
|
+
`VsQueueManager` logs lifecycle events on its own logger (`vs_queue.manager.vs_queue_manager`):
|
|
578
|
+
|
|
579
|
+
| Level | Line |
|
|
580
|
+
|---|---|
|
|
581
|
+
| `INFO` | `Listener registered \| queue=<q> uuid=<uid>` |
|
|
582
|
+
| `INFO` | `Listeners registered \| queue=<q> concurrency=<n>` |
|
|
583
|
+
| `INFO` | `Listener stopped \| queue=<q> uuid=<uid>` |
|
|
584
|
+
| `ERROR` | `Consumer crashed, restarting in <n>s \| queue=<q> attempt=<n> error=<e>` |
|
|
585
|
+
| `CRITICAL` | `Consumer permanently failed after <n> restarts \| queue=<q> error=<e>` |
|
|
586
|
+
|
|
587
|
+
`Subscribed` / `Message received` land only once the loop is actually consuming, so the absence of
|
|
588
|
+
`Subscribed` after `Listener registered` means `subscribe()` is failing before it starts.
|
|
589
|
+
|
|
590
|
+
Turn on the debug lines:
|
|
591
|
+
|
|
592
|
+
```python
|
|
593
|
+
import logging
|
|
594
|
+
logging.getLogger("vs_queue").setLevel(logging.DEBUG)
|
|
595
|
+
```
|
|
596
|
+
|
|
597
|
+
---
|
|
598
|
+
|
|
563
599
|
## Redis Streams Backend
|
|
564
600
|
|
|
565
601
|
`VsRedisQueue` implements the queue using Redis Streams (`XADD` / `XREADGROUP`). Redis Streams provide persistent, ordered, consumer-group-aware message delivery.
|
|
@@ -529,6 +529,42 @@ await queue.publish("orchestrator:tasks", dlq_message)
|
|
|
529
529
|
|
|
530
530
|
---
|
|
531
531
|
|
|
532
|
+
## Logging
|
|
533
|
+
|
|
534
|
+
Both backends log consumer lifecycle and per-message activity on the provider logger
|
|
535
|
+
(`vs_queue.provider.vs_redis_queue` / `vs_queue.provider.vs_rabbitmq_queue`). Every line uses a
|
|
536
|
+
`Label | key=value` format.
|
|
537
|
+
|
|
538
|
+
| Level | Line | When |
|
|
539
|
+
|---|---|---|
|
|
540
|
+
| `INFO` | `Subscribed \| queue=<q> consumer=<ClassName>` | `consumer.start()` has run and the queue / consumer group is ready — the loop is now listening |
|
|
541
|
+
| `INFO` | `Message received \| queue=<q> id=<msg_id> retry_count=<n>` | Each message pulled off the queue, before `handle()` is called |
|
|
542
|
+
| `DEBUG` | `Message processed \| queue=<q> id=<msg_id>` | `handle()` returned and the message was acked |
|
|
543
|
+
| `INFO` | `Unsubscribed \| queue=<q> consumer=<ClassName>` | The subscribe loop exited (graceful shutdown or permanent failure) |
|
|
544
|
+
| `ERROR` | `Message failed, no retry policy \| id=<msg_id> error=<e>` | `handle()` raised and no `VsRetryPolicy` was supplied |
|
|
545
|
+
|
|
546
|
+
`VsQueueManager` logs lifecycle events on its own logger (`vs_queue.manager.vs_queue_manager`):
|
|
547
|
+
|
|
548
|
+
| Level | Line |
|
|
549
|
+
|---|---|
|
|
550
|
+
| `INFO` | `Listener registered \| queue=<q> uuid=<uid>` |
|
|
551
|
+
| `INFO` | `Listeners registered \| queue=<q> concurrency=<n>` |
|
|
552
|
+
| `INFO` | `Listener stopped \| queue=<q> uuid=<uid>` |
|
|
553
|
+
| `ERROR` | `Consumer crashed, restarting in <n>s \| queue=<q> attempt=<n> error=<e>` |
|
|
554
|
+
| `CRITICAL` | `Consumer permanently failed after <n> restarts \| queue=<q> error=<e>` |
|
|
555
|
+
|
|
556
|
+
`Subscribed` / `Message received` land only once the loop is actually consuming, so the absence of
|
|
557
|
+
`Subscribed` after `Listener registered` means `subscribe()` is failing before it starts.
|
|
558
|
+
|
|
559
|
+
Turn on the debug lines:
|
|
560
|
+
|
|
561
|
+
```python
|
|
562
|
+
import logging
|
|
563
|
+
logging.getLogger("vs_queue").setLevel(logging.DEBUG)
|
|
564
|
+
```
|
|
565
|
+
|
|
566
|
+
---
|
|
567
|
+
|
|
532
568
|
## Redis Streams Backend
|
|
533
569
|
|
|
534
570
|
`VsRedisQueue` implements the queue using Redis Streams (`XADD` / `XREADGROUP`). Redis Streams provide persistent, ordered, consumer-group-aware message delivery.
|
|
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "vs-queue"
|
|
7
|
-
version = "0.1.
|
|
7
|
+
version = "0.1.1"
|
|
8
8
|
description = "Pluggable async message queue library for Viveka Sutra — Redis Streams, RabbitMQ, retry, dead letter, and annotation-based consumers"
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
requires-python = ">=3.11"
|
|
@@ -62,6 +62,7 @@ class VsRabbitMQQueue(VsBaseQueue):
|
|
|
62
62
|
async def subscribe(self, queue: str, consumer: VsBaseConsumer, retry_policy: Optional[VsRetryPolicy] = None) -> None:
|
|
63
63
|
q = await self._channel.declare_queue(queue, durable=True)
|
|
64
64
|
await consumer.start()
|
|
65
|
+
_logger.info(f"Subscribed | queue={queue} consumer={type(consumer).__name__}")
|
|
65
66
|
try:
|
|
66
67
|
async with q.iterator() as iterator:
|
|
67
68
|
async for amqp_message in iterator:
|
|
@@ -73,9 +74,11 @@ class VsRabbitMQQueue(VsBaseQueue):
|
|
|
73
74
|
headers=data["headers"],
|
|
74
75
|
payload=data["payload"],
|
|
75
76
|
)
|
|
77
|
+
_logger.info(f"Message received | queue={queue} id={message.id} retry_count={message.retry_count}")
|
|
76
78
|
try:
|
|
77
79
|
await consumer.handle(message)
|
|
78
80
|
await amqp_message.ack()
|
|
81
|
+
_logger.debug(f"Message processed | queue={queue} id={message.id}")
|
|
79
82
|
except Exception as e:
|
|
80
83
|
await consumer.on_error(message, e)
|
|
81
84
|
if retry_policy:
|
|
@@ -89,3 +92,4 @@ class VsRabbitMQQueue(VsBaseQueue):
|
|
|
89
92
|
_logger.error(f"Message failed, no retry policy | id={message.id} error={e}")
|
|
90
93
|
finally:
|
|
91
94
|
await consumer.stop()
|
|
95
|
+
_logger.info(f"Unsubscribed | queue={queue} consumer={type(consumer).__name__}")
|
|
@@ -68,6 +68,7 @@ class VsRedisQueue(VsBaseQueue):
|
|
|
68
68
|
pass
|
|
69
69
|
|
|
70
70
|
await consumer.start()
|
|
71
|
+
_logger.info(f"Subscribed | queue={queue} consumer={type(consumer).__name__}")
|
|
71
72
|
try:
|
|
72
73
|
while True:
|
|
73
74
|
results = await self._client.xreadgroup(
|
|
@@ -89,9 +90,11 @@ class VsRedisQueue(VsBaseQueue):
|
|
|
89
90
|
headers=json.loads(data["headers"]),
|
|
90
91
|
payload=json.loads(data["payload"]),
|
|
91
92
|
)
|
|
93
|
+
_logger.info(f"Message received | queue={queue} id={message.id} retry_count={message.retry_count}")
|
|
92
94
|
try:
|
|
93
95
|
await consumer.handle(message)
|
|
94
96
|
await self._client.xack(queue, group, msg_id)
|
|
97
|
+
_logger.debug(f"Message processed | queue={queue} id={message.id}")
|
|
95
98
|
except Exception as e:
|
|
96
99
|
await consumer.on_error(message, e)
|
|
97
100
|
if retry_policy:
|
|
@@ -102,3 +105,4 @@ class VsRedisQueue(VsBaseQueue):
|
|
|
102
105
|
_logger.error(f"Message failed, no retry policy | id={message.id} error={e}")
|
|
103
106
|
finally:
|
|
104
107
|
await consumer.stop()
|
|
108
|
+
_logger.info(f"Unsubscribed | queue={queue} consumer={type(consumer).__name__}")
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: vs-queue
|
|
3
|
-
Version: 0.1.
|
|
3
|
+
Version: 0.1.1
|
|
4
4
|
Summary: Pluggable async message queue library for Viveka Sutra — Redis Streams, RabbitMQ, retry, dead letter, and annotation-based consumers
|
|
5
5
|
Project-URL: Homepage, https://vivekasutra.com/
|
|
6
6
|
Project-URL: Source, https://github.com/vivekasutra/viveka-mula
|
|
@@ -560,6 +560,42 @@ await queue.publish("orchestrator:tasks", dlq_message)
|
|
|
560
560
|
|
|
561
561
|
---
|
|
562
562
|
|
|
563
|
+
## Logging
|
|
564
|
+
|
|
565
|
+
Both backends log consumer lifecycle and per-message activity on the provider logger
|
|
566
|
+
(`vs_queue.provider.vs_redis_queue` / `vs_queue.provider.vs_rabbitmq_queue`). Every line uses a
|
|
567
|
+
`Label | key=value` format.
|
|
568
|
+
|
|
569
|
+
| Level | Line | When |
|
|
570
|
+
|---|---|---|
|
|
571
|
+
| `INFO` | `Subscribed \| queue=<q> consumer=<ClassName>` | `consumer.start()` has run and the queue / consumer group is ready — the loop is now listening |
|
|
572
|
+
| `INFO` | `Message received \| queue=<q> id=<msg_id> retry_count=<n>` | Each message pulled off the queue, before `handle()` is called |
|
|
573
|
+
| `DEBUG` | `Message processed \| queue=<q> id=<msg_id>` | `handle()` returned and the message was acked |
|
|
574
|
+
| `INFO` | `Unsubscribed \| queue=<q> consumer=<ClassName>` | The subscribe loop exited (graceful shutdown or permanent failure) |
|
|
575
|
+
| `ERROR` | `Message failed, no retry policy \| id=<msg_id> error=<e>` | `handle()` raised and no `VsRetryPolicy` was supplied |
|
|
576
|
+
|
|
577
|
+
`VsQueueManager` logs lifecycle events on its own logger (`vs_queue.manager.vs_queue_manager`):
|
|
578
|
+
|
|
579
|
+
| Level | Line |
|
|
580
|
+
|---|---|
|
|
581
|
+
| `INFO` | `Listener registered \| queue=<q> uuid=<uid>` |
|
|
582
|
+
| `INFO` | `Listeners registered \| queue=<q> concurrency=<n>` |
|
|
583
|
+
| `INFO` | `Listener stopped \| queue=<q> uuid=<uid>` |
|
|
584
|
+
| `ERROR` | `Consumer crashed, restarting in <n>s \| queue=<q> attempt=<n> error=<e>` |
|
|
585
|
+
| `CRITICAL` | `Consumer permanently failed after <n> restarts \| queue=<q> error=<e>` |
|
|
586
|
+
|
|
587
|
+
`Subscribed` / `Message received` land only once the loop is actually consuming, so the absence of
|
|
588
|
+
`Subscribed` after `Listener registered` means `subscribe()` is failing before it starts.
|
|
589
|
+
|
|
590
|
+
Turn on the debug lines:
|
|
591
|
+
|
|
592
|
+
```python
|
|
593
|
+
import logging
|
|
594
|
+
logging.getLogger("vs_queue").setLevel(logging.DEBUG)
|
|
595
|
+
```
|
|
596
|
+
|
|
597
|
+
---
|
|
598
|
+
|
|
563
599
|
## Redis Streams Backend
|
|
564
600
|
|
|
565
601
|
`VsRedisQueue` implements the queue using Redis Streams (`XADD` / `XREADGROUP`). Redis Streams provide persistent, ordered, consumer-group-aware message delivery.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|