async-batcher 0.2.0__tar.gz → 0.2.2__tar.gz

async_batcher-0.2.2/PKG-INFO

@@ -0,0 +1,183 @@
+Metadata-Version: 2.3
+Name: async-batcher
+Version: 0.2.2
+Summary: A service to batch your http requests.
+License: Apache-2.0
+Keywords: python,asyncio,rest,grpc,fastapi
+Author: Hussein Awala
+Author-email: hussein@awala.fr
+Requires-Python: >=3.10,<4.0
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Provides-Extra: aws
+Provides-Extra: keras
+Provides-Extra: scylla
+Provides-Extra: sklearn
+Provides-Extra: sqlalchemy
+Requires-Dist: aioboto3 (>=13.1,<14.0) ; extra == "aws"
+Requires-Dist: keras (>=2.12,<3.0) ; (python_version < "3.12") and (extra == "keras")
+Requires-Dist: keras (>=2.12,<4) ; (python_version == "3.12") and (extra == "keras")
+Requires-Dist: scikit-learn (>=1.2,<2.0) ; (python_version < "3.12") and (extra == "sklearn")
+Requires-Dist: scikit-learn (>=1.3,<2.0) ; (python_version == "3.12") and (extra == "sklearn")
+Requires-Dist: scylla-driver (>=3,<4) ; extra == "scylla"
+Requires-Dist: sqlalchemy[asyncio] (>=1.4,<3) ; extra == "sqlalchemy"
+Project-URL: Bug Tracker, https://github.com/hussein-awala/async-batcher/issues
+Project-URL: Documentation, https://github.com/hussein-awala/async-batcher/blob/main/README.md
+Project-URL: Repository, https://github.com/hussein-awala/async-batcher
+Description-Content-Type: text/markdown
+
+# AsyncBatcher - Asynchronous Batching for Python
+
+## Overview
+AsyncBatcher is a generic, asynchronous batch processor for Python that efficiently groups incoming items into batches
+and processes them asynchronously. It is designed for scenarios where multiple requests or tasks need to be handled in
+batches to improve efficiency and throughput.
+
+## Key Features
+- Asynchronous processing: Uses asyncio for non-blocking execution.
+- Batching mechanism: Groups items into batches based on size or time constraints.
+- Concurrency control: Limits the number of concurrent batch executions.
+- Custom processing logic: Users must implement the `process_batch` method to define batch behavior.
+- Queue management: Uses an `asyncio.Queue` to manage incoming items.
+- Error handling: Ensures robust error reporting and handling.
+
+## How it works
+
+### 1. Receiving Items for Processing
+- Users call `process(item)`, which adds the item to an internal queue.
+- A `Future` object is returned immediately and the result is awaited asynchronously.
+
+### 2. Queue Management and Batching
+- A background task (`run()`) continuously monitors the queue.
+- Items are collected into batches based on:
+  - `max_batch_size`: Maximum items per batch.
+  - `max_queue_time`: Maximum time an item can wait before being processed.
+- Once a batch is ready, it is passed to the processing function.
+
+### 3. Processing the Batch
+- If `process_batch` is asynchronous, it is awaited directly.
+- If `process_batch` is synchronous, it runs inside an `Executor`.
+- Each item’s future is resolved with the corresponding processed result.
+
+### 4. Concurrency Control
+- If `concurrency > 0`, a semaphore ensures that only a limited number of batches are processed simultaneously.
+- Otherwise, all batches run concurrently.
+
+### 5. Stopping the Batcher
+- Calling `stop(force=True)` cancels all ongoing tasks.
+- Calling `stop(force=False)` waits for pending items to be processed before shutting down.
+
+```mermaid
+sequenceDiagram
+    participant User
+    participant AsyncBatcher
+    participant Queue as asyncio.Queue
+    participant RunLoop
+    participant Semaphore
+    participant BatchProcessor
+
+    User->>AsyncBatcher: process(item)
+    activate AsyncBatcher
+    AsyncBatcher->>Queue: put(QueueItem(item, future))
+    AsyncBatcher-->>User: returns future
+    deactivate AsyncBatcher
+
+    Note over AsyncBatcher: Starts RunLoop on first process()
+
+    loop Run Loop (run() method)
+        RunLoop->>Queue: Collect items (max_batch_size/max_queue_time)
+        activate Queue
+        Queue-->>RunLoop: Batch [QueueItem1, QueueItem2...]
+        deactivate Queue
+
+        alt Concurrency Limited (concurrency > 0)
+            RunLoop->>Semaphore: acquire()
+            activate Semaphore
+            Semaphore-->>RunLoop: acquired
+            deactivate Semaphore
+        end
+
+        RunLoop->>BatchProcessor: create_task(_batch_run(batch))
+        activate BatchProcessor
+
+        alt Async process_batch
+            BatchProcessor->>AsyncBatcher: await process_batch(batch)
+        else Sync process_batch
+            BatchProcessor->>Executor: run_in_executor(process_batch)
+        end
+
+        AsyncBatcher-->>BatchProcessor: results [S1, S2...]
+        BatchProcessor->>QueueItem1.future: set_result(S1)
+        BatchProcessor->>QueueItem2.future: set_result(S2)
+        deactivate BatchProcessor
+
+        alt Concurrency Limited
+            RunLoop->>Semaphore: release()
+        end
+    end
+
+    Note over User, BatchProcessor: User's await future gets resolved
+```
+
+## How to use
+
+To use the library, you need to install the package in your environment. You can install the package using pip:
+
+```bash
+pip install async-batcher
+```
+
+Then, you can create a new `AsyncBatcher` class by implementing the `process_batch` method:
+
+```python
+import asyncio
+import logging
+
+from async_batcher.batcher import AsyncBatcher
+
+class MyBatchProcessor(AsyncBatcher[int, int]):
+    async def process_batch(self, batch: list[int]) -> list[int]:
+        await asyncio.sleep(1)  # Simulate processing delay
+        return [x * 2 for x in batch]  # Example: Doubling each item
+
+async def main():
+    batcher = MyBatchProcessor(max_batch_size=5, max_queue_time=2.0, concurrency=2)
+    results = await asyncio.gather(*[batcher.process(i) for i in range(10)])
+    print(results)  # Output: [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
+    await batcher.stop()
+
+# Set logging level to DEBUG if you want to see more details and understand the flow
+logging.basicConfig(level=logging.DEBUG)
+asyncio.run(main())
+```
+
+## Benchmark
+
+The benchmark is available in the [BENCHMARK.md](https://github.com/hussein-awala/async-batcher/blob/main/BENCHMARK.md)
+file.
+
+## When to Use AsyncBatcher?
+The AsyncBatcher library is ideal for applications that need to efficiently handle asynchronous requests in batches,
+such as:
+
+### Machine Learning Model Serving
+- Batch-processing requests to optimize inference performance (e.g., TensorFlow, PyTorch, Scikit-learn).
+
+### Database Bulk Operations
+- Inserting multiple records in a single query to improve I/O efficiency and reduce costs (e.g., PostgreSQL, MySQL,
+  AWS DynamoDB). 
+
+### Messaging and Network Optimization
+- Sending multiple messages in a single API call to reduce latency and costs (e.g., Kafka, RabbitMQ, AWS SQS, AWS SNS).
+
+### Rate-Limited API Calls
+- Aggregating requests to comply with API rate limits (e.g., GitHub API, Twitter API, OpenAI API).
+
+## Final Notes
+- Implement `process_batch` according to your needs.
+- Ensure `max_batch_size` and `max_queue_time` are configured based on performance requirements.
+- Handle exceptions inside `process_batch` to avoid failures affecting other tasks.

async_batcher-0.2.2/README.md

@@ -0,0 +1,151 @@
+# AsyncBatcher - Asynchronous Batching for Python
+
+## Overview
+AsyncBatcher is a generic, asynchronous batch processor for Python that efficiently groups incoming items into batches
+and processes them asynchronously. It is designed for scenarios where multiple requests or tasks need to be handled in
+batches to improve efficiency and throughput.
+
+## Key Features
+- Asynchronous processing: Uses asyncio for non-blocking execution.
+- Batching mechanism: Groups items into batches based on size or time constraints.
+- Concurrency control: Limits the number of concurrent batch executions.
+- Custom processing logic: Users must implement the `process_batch` method to define batch behavior.
+- Queue management: Uses an `asyncio.Queue` to manage incoming items.
+- Error handling: Ensures robust error reporting and handling.
+
+## How it works
+
+### 1. Receiving Items for Processing
+- Users call `process(item)`, which adds the item to an internal queue.
+- A `Future` object is returned immediately and the result is awaited asynchronously.
+
+### 2. Queue Management and Batching
+- A background task (`run()`) continuously monitors the queue.
+- Items are collected into batches based on:
+  - `max_batch_size`: Maximum items per batch.
+  - `max_queue_time`: Maximum time an item can wait before being processed.
+- Once a batch is ready, it is passed to the processing function.
+
+### 3. Processing the Batch
+- If `process_batch` is asynchronous, it is awaited directly.
+- If `process_batch` is synchronous, it runs inside an `Executor`.
+- Each item’s future is resolved with the corresponding processed result.
+
+### 4. Concurrency Control
+- If `concurrency > 0`, a semaphore ensures that only a limited number of batches are processed simultaneously.
+- Otherwise, all batches run concurrently.
+
+### 5. Stopping the Batcher
+- Calling `stop(force=True)` cancels all ongoing tasks.
+- Calling `stop(force=False)` waits for pending items to be processed before shutting down.
+
+```mermaid
+sequenceDiagram
+    participant User
+    participant AsyncBatcher
+    participant Queue as asyncio.Queue
+    participant RunLoop
+    participant Semaphore
+    participant BatchProcessor
+
+    User->>AsyncBatcher: process(item)
+    activate AsyncBatcher
+    AsyncBatcher->>Queue: put(QueueItem(item, future))
+    AsyncBatcher-->>User: returns future
+    deactivate AsyncBatcher
+
+    Note over AsyncBatcher: Starts RunLoop on first process()
+
+    loop Run Loop (run() method)
+        RunLoop->>Queue: Collect items (max_batch_size/max_queue_time)
+        activate Queue
+        Queue-->>RunLoop: Batch [QueueItem1, QueueItem2...]
+        deactivate Queue
+
+        alt Concurrency Limited (concurrency > 0)
+            RunLoop->>Semaphore: acquire()
+            activate Semaphore
+            Semaphore-->>RunLoop: acquired
+            deactivate Semaphore
+        end
+
+        RunLoop->>BatchProcessor: create_task(_batch_run(batch))
+        activate BatchProcessor
+
+        alt Async process_batch
+            BatchProcessor->>AsyncBatcher: await process_batch(batch)
+        else Sync process_batch
+            BatchProcessor->>Executor: run_in_executor(process_batch)
+        end
+
+        AsyncBatcher-->>BatchProcessor: results [S1, S2...]
+        BatchProcessor->>QueueItem1.future: set_result(S1)
+        BatchProcessor->>QueueItem2.future: set_result(S2)
+        deactivate BatchProcessor
+
+        alt Concurrency Limited
+            RunLoop->>Semaphore: release()
+        end
+    end
+
+    Note over User, BatchProcessor: User's await future gets resolved
+```
+
+## How to use
+
+To use the library, you need to install the package in your environment. You can install the package using pip:
+
+```bash
+pip install async-batcher
+```
+
+Then, you can create a new `AsyncBatcher` class by implementing the `process_batch` method:
+
+```python
+import asyncio
+import logging
+
+from async_batcher.batcher import AsyncBatcher
+
+class MyBatchProcessor(AsyncBatcher[int, int]):
+    async def process_batch(self, batch: list[int]) -> list[int]:
+        await asyncio.sleep(1)  # Simulate processing delay
+        return [x * 2 for x in batch]  # Example: Doubling each item
+
+async def main():
+    batcher = MyBatchProcessor(max_batch_size=5, max_queue_time=2.0, concurrency=2)
+    results = await asyncio.gather(*[batcher.process(i) for i in range(10)])
+    print(results)  # Output: [0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
+    await batcher.stop()
+
+# Set logging level to DEBUG if you want to see more details and understand the flow
+logging.basicConfig(level=logging.DEBUG)
+asyncio.run(main())
+```
+
+## Benchmark
+
+The benchmark is available in the [BENCHMARK.md](https://github.com/hussein-awala/async-batcher/blob/main/BENCHMARK.md)
+file.
+
+## When to Use AsyncBatcher?
+The AsyncBatcher library is ideal for applications that need to efficiently handle asynchronous requests in batches,
+such as:
+
+### Machine Learning Model Serving
+- Batch-processing requests to optimize inference performance (e.g., TensorFlow, PyTorch, Scikit-learn).
+
+### Database Bulk Operations
+- Inserting multiple records in a single query to improve I/O efficiency and reduce costs (e.g., PostgreSQL, MySQL,
+  AWS DynamoDB). 
+
+### Messaging and Network Optimization
+- Sending multiple messages in a single API call to reduce latency and costs (e.g., Kafka, RabbitMQ, AWS SQS, AWS SNS).
+
+### Rate-Limited API Calls
+- Aggregating requests to comply with API rate limits (e.g., GitHub API, Twitter API, OpenAI API).
+
+## Final Notes
+- Implement `process_batch` according to your needs.
+- Ensure `max_batch_size` and `max_queue_time` are configured based on performance requirements.
+- Handle exceptions inside `process_batch` to avoid failures affecting other tasks.

{async_batcher-0.2.0 → async_batcher-0.2.2}/async_batcher/batcher.py

@@ -80,8 +80,8 @@ class AsyncBatcher(Generic[T, S], abc.ABC):
         self._current_task: asyncio.Task | None = None
         self._running_batches: dict[int, asyncio.Task] = {}
         self._concurrency_semaphore = asyncio.Semaphore(concurrency) if concurrency > 0 else None
-        self._stop = False
-        self._is_running = False
+        self._stop = asyncio.Event()
+        self._is_running = asyncio.Event()
 
     @abc.abstractmethod
     async def process_batch(self, batch: list[T]) -> list[S] | None:
@@ -99,7 +99,7 @@ class AsyncBatcher(Generic[T, S], abc.ABC):
         Returns:
             S: The result of processing the item.
         """
-        if self._stop:
+        if self._stop.is_set():
             raise RuntimeError("Batcher is stopped")
         if self._current_task is None:
             self._current_task = asyncio.get_running_loop().create_task(self.run())
@@ -118,7 +118,7 @@ class AsyncBatcher(Generic[T, S], abc.ABC):
             return []
         if started_at is None:
             started_at = asyncio.get_event_loop().time()
-        while 1:
+        while True:
             try:
                 max_wait = self.max_queue_time - (asyncio.get_running_loop().time() - started_at)
                 if max_wait > 0:
@@ -166,7 +166,7 @@ class AsyncBatcher(Generic[T, S], abc.ABC):
 
     async def run(self):
         """Run the batcher asynchronously."""
-        self._is_running = True
+        self._is_running.set()
         task_id = 0
         if self.concurrency > 0:
             started_at = None
@@ -204,10 +204,10 @@ class AsyncBatcher(Generic[T, S], abc.ABC):
                         self._batch_run(task_id, batch)
                     )
                     task_id += 1
-        self._is_running = False
+        self._is_running.clear()
 
     def _should_stop(self):
-        return self._stop and self._queue.qsize() == 0
+        return self._stop.is_set() and self._queue.qsize() == 0
 
     async def is_running(self):
         """Check if the batcher is running.
@@ -215,7 +215,7 @@ class AsyncBatcher(Generic[T, S], abc.ABC):
         Returns:
             bool: True if the batcher is running, False otherwise.
         """
-        return self._is_running
+        return self._is_running.is_set()
 
     async def stop(self, force: bool = False, timeout: float | None = None):
         """Stop the batcher asyncio task.
@@ -234,7 +234,7 @@ class AsyncBatcher(Generic[T, S], abc.ABC):
                 if not task.done():
                     task.cancel()
         else:
-            self._stop = True
+            self._stop.set()
             if (
                 self._current_task
                 and not self._current_task.done()

{async_batcher-0.2.0 → async_batcher-0.2.2}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "async-batcher"
-version = "0.2.0"
+version = "0.2.2"
 description = "A service to batch your http requests."
 authors = ["Hussein Awala <hussein@awala.fr>"]
 readme = "README.md"
@@ -15,7 +15,7 @@ packages = [{include = "async_batcher"}]
 
 [tool.poetry.dependencies]
 python = "^3.10"
-aioboto3 = {version = "^12.3", optional = true}
+aioboto3 = {version = "^13.1", optional = true}
 scylla-driver = {version = "^3", optional = true}
 keras = [
     {version = "^2.12", optional = true, markers = "python_version < '3.12'"},
@@ -39,7 +39,7 @@ mock = "^5.1.0"
 pytest = "^8"
 pytest-asyncio = "^0.23.5"
 pre-commit = "^3"
-types-aioboto3 = {version = "^12.3", extras = ["dynamodb"]}
+types-aioboto3 = {version = "^13.1", extras = ["dynamodb"]}
 aiosqlite = "^0.20"
 asyncpg = "^0.29"
 

async_batcher-0.2.0/PKG-INFO

@@ -1,101 +0,0 @@
-Metadata-Version: 2.1
-Name: async-batcher
-Version: 0.2.0
-Summary: A service to batch your http requests.
-Home-page: https://github.com/hussein-awala/async-batcher
-License: Apache-2.0
-Keywords: python,asyncio,rest,grpc,fastapi
-Author: Hussein Awala
-Author-email: hussein@awala.fr
-Requires-Python: >=3.10,<4.0
-Classifier: License :: OSI Approved :: Apache Software License
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Provides-Extra: aws
-Provides-Extra: keras
-Provides-Extra: scylla
-Provides-Extra: sklearn
-Provides-Extra: sqlalchemy
-Requires-Dist: aioboto3 (>=12.3,<13.0) ; extra == "aws"
-Requires-Dist: keras (>=2.12,<3.0) ; (python_version < "3.12") and (extra == "keras")
-Requires-Dist: keras (>=2.12,<4) ; (python_version == "3.12") and (extra == "keras")
-Requires-Dist: scikit-learn (>=1.2,<2.0) ; (python_version < "3.12") and (extra == "sklearn")
-Requires-Dist: scikit-learn (>=1.3,<2.0) ; (python_version == "3.12") and (extra == "sklearn")
-Requires-Dist: scylla-driver (>=3,<4) ; extra == "scylla"
-Requires-Dist: sqlalchemy[asyncio] (>=1.4,<3) ; extra == "sqlalchemy"
-Project-URL: Bug Tracker, https://github.com/hussein-awala/async-batcher/issues
-Project-URL: Documentation, https://github.com/hussein-awala/async-batcher/blob/main/README.md
-Project-URL: Repository, https://github.com/hussein-awala/async-batcher
-Description-Content-Type: text/markdown
-
-# Async Batcher
-
-This project provides a Python library to batch the asynchronous requests and handle them in batches.
-
-## How to use
-
-To use the library, you need to install the package in your environment. You can install the package using pip:
-
-```bash
-pip install async-batcher
-```
-
-Then, you can create a new `AsyncBatcher` class by implementing the `process_batch` method:
-
-```python
-from async_batcher.batcher import AsyncBatcher
-
-class MyAsyncBatcher(AsyncBatcher):
-    async def process_batch(self, batch):
-        # Process the batch
-        print(batch)
-
-# Create a new instance of the `MyAsyncBatcher` class
-async_batcher = MyAsyncBatcher(max_batch_size=20)
-async_batcher.start()
-```
-
-## Benchmark
-
-To evaluate the performance of the `AsyncBatcher` library, we used the [Keras example](examples/keras)
-with [locust](https://locust.io/) to simulate multiple users making requests to the server.
-
-In this example, we have a tensorflow model that takes ~11ms to make a single prediction and ~13ms to process
-a batch of 200 predictions.
-
-For this benchmark, we ran the FastAPI server with Uvicorn with 12 workers,
-on a MacBook Pro with Apple M2 Max chip (12 cores).
-
-Predict endpoint:
-
-![predict__total_requests_per_second.png](assets%2Fpredict__total_requests_per_second.png)
-
-Optimized Predict endpoint:
-
-![optimized_predict__total_requests_per_second.png](assets%2Foptimized_predict__total_requests_per_second.png)
-
-As we can see from the graphs, for the `/predict` endpoint, the failure rate started to increase linearly at the RPS
-of ~284 requests/second, and the 95th percentile response time was ~1100ms at this point. Then with the increase of
-the RPS, the number of successful requests was ~130 requests/second in average, with a high failure rate
-(~300 from ~400 requests/second).
-
-For the `/optimized_predict` endpoint, the failure was smaller than 3 requests/second (0.00625%) during the whole test,
-the average response time was increasing slightly with the increase of the RPS, and it reached ~120ms in average at
-the end of the test (> 480 requests/second), with a 95th percentile response time almost stable and smaller than 500ms.
-
-You can find check the reports for more details:
-- [predict__report.html](assets/predict__report.html)
-- [optimized_predict__report.html](assets/optimized_predict__report.html)
-## Use cases
-
-The `AsyncBatcher` library can be used in any application that needs to handle asynchronous requests in batches,
-such as:
-- Serving machine learning models that optimize the batch processing (e.g. TensorFlow, PyTorch, Scikit-learn, etc.)
-- Storing multiple records in a database in a single query to optimize the I/O operations (or to reduce the cost of the
-  database operations, e.g. AWS DynamoDB)
-- Sending multiple messages in a single request to optimize the network operations (or to reduce the cost of the network
-  operations, e.g. Kafka, RabbitMQ, AWS SQS, AWS SNS, etc.)
-
-

async_batcher-0.2.0/README.md

@@ -1,68 +0,0 @@
-# Async Batcher
-
-This project provides a Python library to batch the asynchronous requests and handle them in batches.
-
-## How to use
-
-To use the library, you need to install the package in your environment. You can install the package using pip:
-
-```bash
-pip install async-batcher
-```
-
-Then, you can create a new `AsyncBatcher` class by implementing the `process_batch` method:
-
-```python
-from async_batcher.batcher import AsyncBatcher
-
-class MyAsyncBatcher(AsyncBatcher):
-    async def process_batch(self, batch):
-        # Process the batch
-        print(batch)
-
-# Create a new instance of the `MyAsyncBatcher` class
-async_batcher = MyAsyncBatcher(max_batch_size=20)
-async_batcher.start()
-```
-
-## Benchmark
-
-To evaluate the performance of the `AsyncBatcher` library, we used the [Keras example](examples/keras)
-with [locust](https://locust.io/) to simulate multiple users making requests to the server.
-
-In this example, we have a tensorflow model that takes ~11ms to make a single prediction and ~13ms to process
-a batch of 200 predictions.
-
-For this benchmark, we ran the FastAPI server with Uvicorn with 12 workers,
-on a MacBook Pro with Apple M2 Max chip (12 cores).
-
-Predict endpoint:
-
-![predict__total_requests_per_second.png](assets%2Fpredict__total_requests_per_second.png)
-
-Optimized Predict endpoint:
-
-![optimized_predict__total_requests_per_second.png](assets%2Foptimized_predict__total_requests_per_second.png)
-
-As we can see from the graphs, for the `/predict` endpoint, the failure rate started to increase linearly at the RPS
-of ~284 requests/second, and the 95th percentile response time was ~1100ms at this point. Then with the increase of
-the RPS, the number of successful requests was ~130 requests/second in average, with a high failure rate
-(~300 from ~400 requests/second).
-
-For the `/optimized_predict` endpoint, the failure was smaller than 3 requests/second (0.00625%) during the whole test,
-the average response time was increasing slightly with the increase of the RPS, and it reached ~120ms in average at
-the end of the test (> 480 requests/second), with a 95th percentile response time almost stable and smaller than 500ms.
-
-You can find check the reports for more details:
-- [predict__report.html](assets/predict__report.html)
-- [optimized_predict__report.html](assets/optimized_predict__report.html)
-## Use cases
-
-The `AsyncBatcher` library can be used in any application that needs to handle asynchronous requests in batches,
-such as:
-- Serving machine learning models that optimize the batch processing (e.g. TensorFlow, PyTorch, Scikit-learn, etc.)
-- Storing multiple records in a database in a single query to optimize the I/O operations (or to reduce the cost of the
-  database operations, e.g. AWS DynamoDB)
-- Sending multiple messages in a single request to optimize the network operations (or to reduce the cost of the network
-  operations, e.g. Kafka, RabbitMQ, AWS SQS, AWS SNS, etc.)
-