async-lambda-unstable 0.3.12__tar.gz → 0.4.3__tar.gz

{async-lambda-unstable-0.3.12 → async-lambda-unstable-0.4.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: async-lambda-unstable
-Version: 0.3.12
+Version: 0.4.3
 Summary: A framework for creating AWS Lambda Async Workflows. - Unstable Branch
 Author-email: "Nuclei, Inc" <engineering@nuclei.ai>
 Description-Content-Type: text/markdown
@@ -186,8 +186,83 @@ def api_task(event: APIEvent):
     event.headers # request headers
     event.querystring_params # request querystring params
     event.body # request body
+    event.headers # This is a case insensitive dict
 ```
 
+# Middleware
+
+Middleware functions can be registered with controllers which will wrap the execution of tasks.
+These functions can be configured to trigger on specific types of tasks and can trigger
+side effects and modify the `event` or `response` objects.
+
+Middleware functions must have the signature `Callable[[BaseEvent, Callable[[BaseEvent], T]], T]`.
+The first argument is the `event`, and the second argument (`call_next`) is a function which will propagate the
+calls down the middleware/task stack. The `call_next` function must be called, and its result in most cases be returned.
+If this is not done then tasks will not run as expected.
+
+**Extreme care should be taken with middleware as a simple mistake can have catastrophic effects.**
+
+- Middleware functions are run in the order which they were registered and parent controller middleware will be run first.
+
+- Middleware functions which are registered more than once will only be run once.
+
+Registration can be done when the `AsyncLambdaController` is initialized with the parameter `middleware` or by using the `add_middleware` method.
+
+Middleware functions have three sections:
+
+1. Pre task
+2. Task execution
+3. Post task
+
+```python
+def async_lambda_middleware(event: BaseEvent, call_next):
+    # pre task
+    result = call_next(event) # task execution
+    # post task
+    return result
+```
+
+If there are multiple middleware functions then `call_next` will actually be calling the next middleware function in the stack.
+
+For example if there is middleware functions `A` and `B` registered in that order.
+Then the execution order would go:
+
+`A(Pre)` -> `B(Pre)` -> `Task` -> `B(Post)` -> `A(Post)`
+
+EX:
+
+```python
+def async_task_only_middleware(event: ManagedSQSEvent, call_next):
+    print(f"Invocation Payload: {event}")
+    result = call_next(event)
+    print(f"Invocation Result: {result}")
+    return result
+
+def all_task_types_middleware(event: BaseEvent, call_next):
+    print(f"This event is of the type {type(event)}")
+    result = call_next(event)
+    print(f"The result is of the type {type(result)}")
+    return event
+
+controller = AsyncLambdaController(middleware=[([BaseEvent], all_task_types_middleware)])
+
+controller.add_middleware([ManagedSQSEvent], async_task_only_middleware)
+
+@controller.async_task("ATask")
+def a_task(event: ManagedSQSEvent):
+    pass
+
+@controller.api_task("BTask", "/test", "get")
+def b_task(event: APIEvent):
+    return "hello world"
+```
+
+In this scenario when `ATask` is invoked first `all_task_types_middleware` will be run, then
+`async_task_only_middleware` will be run and finally the `a_task` function will be executed.
+
+When `BTask` is invoked first `all_task_types_middleware` will be run, and then the `b_task`
+function will be executed
+
 # `async-lambda` config
 
 Configuration options can be set with the `.async_lambda/config.json` file.

{async-lambda-unstable-0.3.12 → async-lambda-unstable-0.4.3}/README.md

@@ -178,8 +178,83 @@ def api_task(event: APIEvent):
     event.headers # request headers
     event.querystring_params # request querystring params
     event.body # request body
+    event.headers # This is a case insensitive dict
 ```
 
+# Middleware
+
+Middleware functions can be registered with controllers which will wrap the execution of tasks.
+These functions can be configured to trigger on specific types of tasks and can trigger
+side effects and modify the `event` or `response` objects.
+
+Middleware functions must have the signature `Callable[[BaseEvent, Callable[[BaseEvent], T]], T]`.
+The first argument is the `event`, and the second argument (`call_next`) is a function which will propagate the
+calls down the middleware/task stack. The `call_next` function must be called, and its result in most cases be returned.
+If this is not done then tasks will not run as expected.
+
+**Extreme care should be taken with middleware as a simple mistake can have catastrophic effects.**
+
+- Middleware functions are run in the order which they were registered and parent controller middleware will be run first.
+
+- Middleware functions which are registered more than once will only be run once.
+
+Registration can be done when the `AsyncLambdaController` is initialized with the parameter `middleware` or by using the `add_middleware` method.
+
+Middleware functions have three sections:
+
+1. Pre task
+2. Task execution
+3. Post task
+
+```python
+def async_lambda_middleware(event: BaseEvent, call_next):
+    # pre task
+    result = call_next(event) # task execution
+    # post task
+    return result
+```
+
+If there are multiple middleware functions then `call_next` will actually be calling the next middleware function in the stack.
+
+For example if there is middleware functions `A` and `B` registered in that order.
+Then the execution order would go:
+
+`A(Pre)` -> `B(Pre)` -> `Task` -> `B(Post)` -> `A(Post)`
+
+EX:
+
+```python
+def async_task_only_middleware(event: ManagedSQSEvent, call_next):
+    print(f"Invocation Payload: {event}")
+    result = call_next(event)
+    print(f"Invocation Result: {result}")
+    return result
+
+def all_task_types_middleware(event: BaseEvent, call_next):
+    print(f"This event is of the type {type(event)}")
+    result = call_next(event)
+    print(f"The result is of the type {type(result)}")
+    return event
+
+controller = AsyncLambdaController(middleware=[([BaseEvent], all_task_types_middleware)])
+
+controller.add_middleware([ManagedSQSEvent], async_task_only_middleware)
+
+@controller.async_task("ATask")
+def a_task(event: ManagedSQSEvent):
+    pass
+
+@controller.api_task("BTask", "/test", "get")
+def b_task(event: APIEvent):
+    return "hello world"
+```
+
+In this scenario when `ATask` is invoked first `all_task_types_middleware` will be run, then
+`async_task_only_middleware` will be run and finally the `a_task` function will be executed.
+
+When `BTask` is invoked first `all_task_types_middleware` will be run, and then the `b_task`
+function will be executed
+
 # `async-lambda` config
 
 Configuration options can be set with the `.async_lambda/config.json` file.

{async-lambda-unstable-0.3.12 → async-lambda-unstable-0.4.3}/async_lambda/__init__.py

@@ -7,9 +7,10 @@ from .env import enable_force_sync_mode as enable_force_sync_mode
 from .env import is_build_mode as is_build_mode
 from .models.case_insensitive_dict import CaseInsensitiveDict as CaseInsensitiveDict
 from .models.events.api_event import APIEvent as APIEvent
+from .models.events.base_event import BaseEvent as BaseEvent
 from .models.events.dynamodb_event import DynamoDBEvent as DynamoDBEvent
 from .models.events.managed_sqs_event import ManagedSQSEvent as ManagedSQSEvent
 from .models.events.scheduled_event import ScheduledEvent as ScheduledEvent
 from .models.events.unmanaged_sqs_event import UnmanagedSQSEvent as UnmanagedSQSEvent
 
-__version__ = "0.3.12"
+__version__ = "0.4.3"

{async-lambda-unstable-0.3.12 → async-lambda-unstable-0.4.3}/async_lambda/controller.py

@@ -4,13 +4,15 @@ import logging
 import re
 import time
 from datetime import datetime, timezone
-from typing import Any, Callable, Dict, Optional, Tuple
+from typing import Any, Callable, Dict, List, Optional, Tuple, Type
 from uuid import uuid4
 
 from . import env
 from .build_config import get_build_config_for_stage
 from .client import get_s3_client, get_sqs_client
+from .middleware import MET, RT, MiddlewareFunction, MiddlewareRegistration
 from .models.events.api_event import APIEvent
+from .models.events.base_event import BaseEvent
 from .models.events.dynamodb_event import DynamoDBEvent
 from .models.events.managed_sqs_event import ManagedSQSEvent
 from .models.events.scheduled_event import ScheduledEvent
@@ -32,6 +34,8 @@ class AsyncLambdaController:
     current_lane: Optional[int] = None
     current_invocation_id: Optional[str] = None
     parent_controller: Optional["AsyncLambdaController"] = None
+    middleware: List[MiddlewareRegistration]
+
     dlq_task_id: Optional[str] = None
 
     def __init__(
@@ -39,11 +43,34 @@ class AsyncLambdaController:
         is_sub: bool = False,
         lane_count: Optional[int] = None,
         propagate_lane_assignment: Optional[bool] = None,
+        middleware: Optional[List[MiddlewareRegistration]] = None,
+        exception_handler: Optional[Callable[[Exception], None]] = None,
     ):
         self.tasks = dict()
         self.is_sub = is_sub
         self.lane_count = lane_count
         self.propagate_lane_assignment = propagate_lane_assignment
+        self.middleware = middleware or list()
+        self.exception_handler = exception_handler
+
+    def add_middleware(
+        self, event_types: List[Type[BaseEvent]], func: MiddlewareFunction[MET, RT]
+    ):
+        self.middleware.append((event_types, func))
+
+    def get_middleware_for_event(self, event: MET) -> List[MiddlewareFunction[MET, RT]]:
+        if self.parent_controller is not None:
+            _middleware_functions = self.parent_controller.get_middleware_for_event(
+                event
+            )
+        else:
+            _middleware_functions = list()
+
+        for event_types, func in self.middleware:
+            if any(isinstance(event, event_type) for event_type in event_types):
+                _middleware_functions.append(func)
+
+        return _middleware_functions
 
     def add_task(self, task: AsyncLambdaTask):
         """
@@ -382,7 +409,7 @@ class AsyncLambdaController:
         """
         Invoke an Async-Lambda task.
         """
-        self._async_invoke(
+        return self._async_invoke(
             destination_task_id=task_id,
             payload=payload,
             delay=delay,

async-lambda-unstable-0.4.3/async_lambda/middleware.py

@@ -0,0 +1,30 @@
+from typing import Callable, Generic, List, Tuple, Type, TypeVar
+
+from .models.events.base_event import BaseEvent
+
+MET = TypeVar("MET", bound=BaseEvent)
+RT = TypeVar("RT")
+
+MiddlewareFunction = Callable[[MET, Callable[[MET], RT]], RT]
+MiddlewareRegistration = Tuple[List[Type[MET]], MiddlewareFunction[MET, RT]]
+
+
+class MiddlewareStackExecutor(Generic[MET, RT]):
+    def __init__(
+        self,
+        middleware: List[MiddlewareFunction],
+        final: Callable[[MET], RT],
+    ):
+        self.middleware = middleware.copy()
+        self.final = final
+        self._ran_fns = list()
+
+    def call_next(self, event: MET) -> RT:
+        while True:
+            if len(self.middleware) == 0:
+                return self.final(event)
+            next_fn = self.middleware.pop(0)
+            if next_fn in self._ran_fns:
+                continue
+            self._ran_fns.append(next_fn)
+            return next_fn(event, self.call_next)

{async-lambda-unstable-0.3.12 → async-lambda-unstable-0.4.3}/async_lambda/models/task.py

@@ -13,6 +13,7 @@ from ..config import config
 if TYPE_CHECKING:
     from ..controller import AsyncLambdaController  # pragma: not covered
 
+from ..middleware import RT, MiddlewareStackExecutor
 from .events.dynamodb_event import DynamoDBEvent
 from .events.managed_sqs_event import ManagedSQSEvent
 from .events.scheduled_event import ScheduledEvent
@@ -33,7 +34,7 @@ EventType = TypeVar(
 )
 
 
-class AsyncLambdaTask(Generic[EventType]):
+class AsyncLambdaTask(Generic[EventType, RT]):
     controller: "AsyncLambdaController"
     task_id: str
     trigger_type: TaskTriggerType
@@ -46,12 +47,12 @@ class AsyncLambdaTask(Generic[EventType]):
     init_tasks: List[Union[Callable[[str], Any], Callable[[], Any]]]
     _has_run_init_tasks: bool
 
-    executable: Callable[[EventType], Any]
+    executable: Callable[[EventType], RT]
 
     def __init__(
         self,
         controller: "AsyncLambdaController",
-        executable: Callable[[EventType], Any],
+        executable: Callable[[EventType], RT],
         task_id: str,
         trigger_type: TaskTriggerType,
         trigger_config: Optional[dict] = None,
@@ -481,9 +482,13 @@ class AsyncLambdaTask(Generic[EventType]):
             return self.controller.get_task(self.trigger_config["dlq_task_id"])
         return self.controller.get_dlq_task()
 
-    def execute(self, event: EventType) -> Any:
+    def execute(self, event: EventType) -> RT:
         """
         Executes the tasks function
         """
         self._run_init_tasks()
-        return self.executable(event)
+        middleware = self.controller.get_middleware_for_event(event)
+        middleware_stack_executor = MiddlewareStackExecutor[EventType, RT](
+            middleware=middleware, final=self.executable
+        )
+        return middleware_stack_executor.call_next(event)

{async-lambda-unstable-0.3.12 → async-lambda-unstable-0.4.3}/async_lambda_unstable.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: async-lambda-unstable
-Version: 0.3.12
+Version: 0.4.3
 Summary: A framework for creating AWS Lambda Async Workflows. - Unstable Branch
 Author-email: "Nuclei, Inc" <engineering@nuclei.ai>
 Description-Content-Type: text/markdown
@@ -186,8 +186,83 @@ def api_task(event: APIEvent):
     event.headers # request headers
     event.querystring_params # request querystring params
     event.body # request body
+    event.headers # This is a case insensitive dict
 ```
 
+# Middleware
+
+Middleware functions can be registered with controllers which will wrap the execution of tasks.
+These functions can be configured to trigger on specific types of tasks and can trigger
+side effects and modify the `event` or `response` objects.
+
+Middleware functions must have the signature `Callable[[BaseEvent, Callable[[BaseEvent], T]], T]`.
+The first argument is the `event`, and the second argument (`call_next`) is a function which will propagate the
+calls down the middleware/task stack. The `call_next` function must be called, and its result in most cases be returned.
+If this is not done then tasks will not run as expected.
+
+**Extreme care should be taken with middleware as a simple mistake can have catastrophic effects.**
+
+- Middleware functions are run in the order which they were registered and parent controller middleware will be run first.
+
+- Middleware functions which are registered more than once will only be run once.
+
+Registration can be done when the `AsyncLambdaController` is initialized with the parameter `middleware` or by using the `add_middleware` method.
+
+Middleware functions have three sections:
+
+1. Pre task
+2. Task execution
+3. Post task
+
+```python
+def async_lambda_middleware(event: BaseEvent, call_next):
+    # pre task
+    result = call_next(event) # task execution
+    # post task
+    return result
+```
+
+If there are multiple middleware functions then `call_next` will actually be calling the next middleware function in the stack.
+
+For example if there is middleware functions `A` and `B` registered in that order.
+Then the execution order would go:
+
+`A(Pre)` -> `B(Pre)` -> `Task` -> `B(Post)` -> `A(Post)`
+
+EX:
+
+```python
+def async_task_only_middleware(event: ManagedSQSEvent, call_next):
+    print(f"Invocation Payload: {event}")
+    result = call_next(event)
+    print(f"Invocation Result: {result}")
+    return result
+
+def all_task_types_middleware(event: BaseEvent, call_next):
+    print(f"This event is of the type {type(event)}")
+    result = call_next(event)
+    print(f"The result is of the type {type(result)}")
+    return event
+
+controller = AsyncLambdaController(middleware=[([BaseEvent], all_task_types_middleware)])
+
+controller.add_middleware([ManagedSQSEvent], async_task_only_middleware)
+
+@controller.async_task("ATask")
+def a_task(event: ManagedSQSEvent):
+    pass
+
+@controller.api_task("BTask", "/test", "get")
+def b_task(event: APIEvent):
+    return "hello world"
+```
+
+In this scenario when `ATask` is invoked first `all_task_types_middleware` will be run, then
+`async_task_only_middleware` will be run and finally the `a_task` function will be executed.
+
+When `BTask` is invoked first `all_task_types_middleware` will be run, and then the `b_task`
+function will be executed
+
 # `async-lambda` config
 
 Configuration options can be set with the `.async_lambda/config.json` file.

{async-lambda-unstable-0.3.12 → async-lambda-unstable-0.4.3}/async_lambda_unstable.egg-info/SOURCES.txt

@@ -8,6 +8,7 @@ async_lambda/config.py
 async_lambda/controller.py
 async_lambda/defer.py
 async_lambda/env.py
+async_lambda/middleware.py
 async_lambda/py.typed
 async_lambda/util.py
 async_lambda/models/__init__.py