qena-shared-lib 0.1.18__tar.gz → 0.1.20__tar.gz

qena_shared_lib-0.1.18/README.md → qena_shared_lib-0.1.20/PKG-INFO

@@ -1,26 +1,105 @@
+Metadata-Version: 2.3
+Name: qena-shared-lib
+Version: 0.1.20
+Summary: A shared tools for other services
+Requires-Dist: fastapi[all]==0.115.6
+Requires-Dist: prometheus-client==0.21.1
+Requires-Dist: prometheus-fastapi-instrumentator==7.0.2
+Requires-Dist: punq==0.7.0
+Requires-Dist: pydantic-core==2.27.1
+Requires-Dist: pydantic==2.10.3
+Requires-Dist: starlette==0.41.3
+Requires-Dist: typing-extensions==4.12.2
+Requires-Dist: aiokafka==0.12.0 ; extra == 'all'
+Requires-Dist: cronsim==2.6 ; extra == 'all'
+Requires-Dist: jwt==1.3.1 ; extra == 'all'
+Requires-Dist: passlib[bcrypt]==1.7.4 ; extra == 'all'
+Requires-Dist: pika==1.3.2 ; extra == 'all'
+Requires-Dist: pymongo==4.15.3 ; extra == 'all'
+Requires-Dist: redis==7.1.0 ; extra == 'all'
+Requires-Dist: aiokafka==0.12.0 ; extra == 'kafka'
+Requires-Dist: pymongo==4.15.3 ; extra == 'mongodb'
+Requires-Dist: pika==1.3.2 ; extra == 'rabbitmq'
+Requires-Dist: redis==7.1.0 ; extra == 'redis'
+Requires-Dist: cronsim==2.6 ; extra == 'scheduler'
+Requires-Dist: jwt==1.3.1 ; extra == 'security'
+Requires-Dist: passlib[bcrypt]==1.7.4 ; extra == 'security'
+Requires-Python: >=3.10
+Provides-Extra: all
+Provides-Extra: kafka
+Provides-Extra: mongodb
+Provides-Extra: rabbitmq
+Provides-Extra: redis
+Provides-Extra: scheduler
+Provides-Extra: security
+Description-Content-Type: text/markdown
+
 # Qena shared lib
 
 A shared tools for other services. It includes.
 
 - FastAPI app builder
 - A wrapper around fastapi to make it class based.
-- RabbitMQ utility class to listen, respond, publish and make rpc request.
+- RabbitMQ utility class to listen , respond , publish and make rpc request.
 - Remote logging
     - Logstash utility class to log message in `ecs` ( elastic common schema ).
-- A simple task scheduler, to schedule task to run in specific time.
+- A simple task scheduler , to schedule task to run in specific time.
 - Background task runner.
-- Security tools ( password hasher, jwt, acl ).
-- IOC container to manager dependencies used across fastapi, rabbitmq manager and schedule manager.
+- Security tools ( password hasher , jwt , acl ).
+- IOC container to manager dependencies used across fastapi , rabbitmq manager and schedule manager.
+- Kafka producer and consumer wrapper.
+- Mongodb client wrapper , with repository and index manager.
+- Redis wrapper with cache and distributed lock manager.
+
+## Installation
 
-# Usage
+It is prefered to use [astral.sh / uv](https://docs.astral.sh/uv) as a package manager.
 
-## Environment variables
+``` sh
+$ uv add qena-shared-lib[all]
+```
+to install all extras , or specific extras `kafka` , `rabbitmq` , `scheduler` , `security` , `redis` or `mongodb`.
+
+## Usage
+
+- [Environment variables](#environment-variables)
+- [Http](#http)
+- [Lifespan](#lifespan)
+- [Dependencies](#dependencies)
+- [Controllers](#controllers)
+- [Routers](#routers)
+- [Remote logging](#remote-logging)
+    - [Logstash](#logstash)
+- [Rabbitmq](#rabbitmq)
+    - [Publisher](#publisher)
+    - [RPC client](#rpc-client)
+    - [Flow control](#flow-control)
+    - [Rpc reply](#rpc-reply)
+    - [Retry consumer](#retry-consumer)
+- [Scheduler](#scheduler)
+- [Background](#background)
+- [Security](#security)
+    - [Password hasher](#password-hasher)
+    - [JWT](#jwt)
+    - [ACL](#acl)
+- [Kafka](#kafka)
+    - [Producer](#producer)
+    - [Consumer](#consumer)
+- [Mongodb](#mongodb)
+    - [Aggregation](#aggregation)
+    - [Index](#index)
+    - [Crud](#crud)
+- [Redis](#redis)
+    - [Cache](#cache)
+    - [Distribute lock](#distribute-lock)
+
+### Environment variables
 
 - `QENA_SHARED_LIB_LOGGING_LOGGER_NAME` root logger name.
 - `QENA_SHARED_LIB_SECURITY_UNAUTHORIZED_RESPONSE_CODE` an integer response on an authorized access of resource.
 - `QENA_SHARED_LIB_SECURITY_TOKEN_HEADER` to header key for jwt token.
 
-## Http
+### Http
 
 To create fastapi app.
 
@@ -160,9 +239,10 @@ def main() -> FastAPI:
     ...
 ```
 
-## Remote logging
+### Remote logging
+
+#### Logstash
 
-### Logstash
 ``` py
 from qena_shared_lib.remotelogging import BaseRemoteLogSender
 from qena_shared_lib.remotelogging.logstash import HTTPSender, # TCPSender
@@ -213,7 +293,7 @@ def log_message(
     remote_logger.info(message)
 ```
 
-## Rabbitmq
+### Rabbitmq
 
 To create rabbitmq connection manager.
 
@@ -261,7 +341,7 @@ def main() -> FastAPI:
     ...
 ```
 
-### Publisher
+#### Publisher
 
 ``` py
 @router.post("")
@@ -278,7 +358,7 @@ async def store_user(
     # await publisher.publish_as_arguments(user)
 ```
 
-### RPC client
+#### RPC client
 
 ``` py
 @router.get("")
@@ -297,7 +377,7 @@ async def get_user(
     return user
 ```
 
-### Flow control
+#### Flow control
 
 ``` py
 from qena_shared_lib.rabbitmq import ... , ListenerContext
@@ -316,7 +396,7 @@ class UserConsumer(ListenerBase):
 
 ```
 
-### Rpc reply
+#### Rpc reply
 
 Optionally it is possible to reply to rpc calls, through.
 
@@ -336,7 +416,7 @@ class UserWorker(ListenerBase):
         ...
 ```
 
-### Retry consumer
+#### Retry consumer
 
 Consumer can retry to consumer a message in an event of failure.
 
@@ -410,7 +490,7 @@ def main() -> FastAPI:
 
 
 
-## Scheduler
+### Scheduler
 
 ``` py
 from qena_shared_lib.scheduler import (
@@ -471,7 +551,7 @@ def main() -> FastAPI:
     ...
 ```
 
-## Background
+### Background
 
 ``` py
 from qena_shared_lib.background import Background
@@ -515,9 +595,9 @@ async def process_data(
     background.add_task(BackgroundTask(data_processor, data))
 ```
 
-## Security
+### Security
 
-### Password hasher
+#### Password hasher
 
 ``` py
 from qena_shared_lib.security import PasswordHasher
@@ -549,7 +629,7 @@ def main() -> FastAPI:
     ...
 ```
 
-### JWT
+#### JWT
 
 ``` py
 from qena_shared_lib.security import JwtAdapter
@@ -591,7 +671,7 @@ def main() -> FastAPI:
     ...
 ```
 
-### ACL
+#### ACL
 
 ``` py
 from qena_shared_lib.security import Authorization
@@ -625,3 +705,313 @@ async def get_users(
 )
     ...
 ```
+
+### Kafka
+
+``` py
+from qena_shared_lib.kafka import KafkaManager
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    kafka = get_service(KafkaManager)
+
+    await kafka.connect()
+
+    yield
+
+    await kafka.disconnect()
+
+
+def main() -> FastAPI:
+    ...
+
+    kafka = KafkaManager(
+        remote_logger=...,
+        bootstrap_servers="127.0.0.1:9092",
+    )
+
+    builder.with_singleton(
+        service=KafkaManager,
+        instance=kafka,
+    )
+
+    ...
+```
+
+#### Producer
+
+``` py
+class UserService:
+    def __init__(self, kafka: KafkaManager):
+        self._kafka = kafka
+
+    async def create_user(self):
+        ...
+
+        async with await self._kafka.producer("user") as producer:
+            await producer.send(key="some_key", value=user)
+
+        ...
+```
+
+#### Consumer
+
+``` py
+from qena_shared_lib.kafka import (
+    ConsumerBase,
+    consume,
+    consumer,
+)
+
+
+@consumer(["user"])
+class USerConsumer(ConsumerBase):
+    def __init__(self, user_service: UserService):
+        self._user_service = user_service
+
+    @consume()
+    async def user_created(self, key: Any | None, value: Any | None):
+        ...
+
+        await self._user_service.create_user(...)
+
+        ...
+```
+
+### Mongodb
+
+``` py
+from qena_shared_lib.mongodb import MongoDBManager
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    db = get_service(MongoDBManager)
+
+    await db.connect()
+
+    yield
+
+    await db.disconnect()
+
+
+def main() -> FastAPI:
+    ...
+
+    db = MongoDBManager(
+        connection_string="mongodb://127.0.0.1:27017",
+        db="userDb"
+    )
+
+    builder.with_singleton(
+        service=MongoDBManager,
+        instance=db
+    )
+
+    ...
+```
+
+#### Crud
+
+``` py
+from qena_shared_lib.mongodb import (
+    Document,
+    MongoDBManager,
+    ProjectedDocument,
+    RepositoryBase,
+)
+
+
+class User(Document):
+    __collection_name__ = "users"
+
+    full_name: str
+    phone: str
+
+
+class FullNameProjectedUser(ProjectedDocument):
+    full_name: str
+
+
+class UserRepository(RepositoryBase[User]):
+    pass
+
+
+class UserService:
+    def __init__(self, user_repository: UserRepository):
+        self._user_repository = user_repository
+
+    async def add_user(self):
+        await self._user_repository.insert(
+            User(
+                full_name="user one",
+                phone="+251900000000"
+            )
+        )
+
+    async def get_user(self):
+        user = await self._user_repository.find_by_filter(
+            filter={"phone": "+251900000000"}
+        )
+
+    async def get_user_fullname(self):
+        user = await self._user_repository.find_by_filter(
+            filter={"phone": "+251900000000"}, projection=FullNameProjectedUser
+        )
+
+    async def update_user(self):
+        user = await self._user_repository.find_by_filter(
+            filter={"phone": "+251900000000"}, projection
+        )
+        user.phone = "+251900000001"
+
+        await user_repository.replace(user)
+```
+
+#### Aggregation
+
+``` py
+class AggregatedUser(AggregatedDocument):
+    __pipeline__ = [
+        {"$match": {"phone": {"$in": ["+251900000000", "+251900000001"]}}},
+        {"$project": {"fullName": True}},
+    ]
+
+    full_name: str
+
+class UserService:
+    ...
+
+    async def get_user_fullnames(self):
+        users = user_repository.aggregate(aggregation=AggregatedUser)
+
+    ...
+```
+
+#### Index
+
+``` py
+from qena_shared_lib.mongodb import Document, IndexManager, IndexModel
+
+
+class User(Document):
+    __collection_name__ = "users"
+    __indexes__ = [IndexModel("phone")]
+
+    full_name: str
+    phone: str
+
+
+async def manage_indexes():
+    ...
+
+    index_manager = IndexManager(db=db, documents=[User])
+
+    await index_manager.create_indexes
+
+    ...
+
+    await index_manager.drop_indexes()
+
+    ...
+```
+
+### Redis
+
+``` py
+from qena_shared_lib.redis import RedisManager
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    redis_manager = get_service(RedisManager)
+
+    await dredis_managerb.connect()
+
+    yield
+
+    await redis_manager.disconnect()
+
+
+def main() -> FastAPI:
+    ...
+
+    redis_manager = RedisManager("redis://127.0.0.1:6379")
+
+    builder.with_singleton(
+        service=RedisManager,
+        instance=redis_manager
+    )
+
+    ...
+```
+
+#### Cache
+
+``` py
+from qena_shared_lib.cache import CachedObject, CacheManager
+
+
+def main() -> FastAPI:
+    ...
+
+    cache_manager = CacheManager()
+
+    redis_manager.add(cache_manager)
+    builder.with_singleton(
+        service=CacheManager,
+        instance=cache_manager
+    )
+
+    ...
+
+
+class UserCache(CachedObject):
+    full_name: str
+
+
+class UserService:
+    def __init__(self, cache_manager: CacheManager):
+        self._cache_manager = cache_manager
+
+    async def cache_user(self):
+        await self._cache_manager.set(
+            UserCache(full_name="user one")
+        )
+
+    async def get_cached_user(self):
+        user_cache = await self._cache_manager.get(UserCache)
+
+    async def unset_cached_user(self):
+        await self._cache_manager.unset(UserCache)
+```
+
+#### Distribute lock
+
+``` py
+from qena_shared_lib.sync import DistributedLockManager
+
+
+def main() -> FastAPI:
+    ...
+
+    distributed_lock_manager = DistributedLockManager()
+
+    redis_manager.add(distributed_lock_manager)
+    builder.with_singleton(
+        service=DistributedLockManager,
+        instance=distributed_lock_manager
+    )
+
+    ...
+
+
+class UserService:
+    def __init__(self, distributed_lock_manager: DistributedLockManager):
+        self._distributed_lock_manager = distributed_lock_manager
+
+    async def create_user(self):
+        async with self._distributed_lock_manager("user_one_create") as _:
+            ...
+```