queuerPy 0.15.0__tar.gz → 0.17.0__tar.gz

{queuerpy-0.15.0/queuerPy.egg-info → queuerpy-0.17.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: queuerPy
-Version: 0.15.0
+Version: 0.17.0
 Summary: A Python implementation of the queuer system - a job queuing and processing system with PostgreSQL backend
 Author-email: Simon Herrmann <siherrmann@users.noreply.github.com>
 Maintainer-email: Simon Herrmann <siherrmann@users.noreply.github.com>
@@ -43,9 +43,10 @@ Dynamic: license-file
 
 # queuerPy
 
+[![PyPi](https://pythonico.leapcell.app/pypi/queuerPy.svg?style=shields&data=n,v,d)](https://pypi.org/project/queuerPy/)
 [![Python](https://img.shields.io/badge/Python-3.10+-blue.svg)](https://www.python.org/)
-[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/siherrmann/queuer/blob/master/LICENSE)
 ![Coverage](https://raw.githubusercontent.com/siherrmann/queuerPy/refs/heads/main/coverage-badge.svg)
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/siherrmann/queuer/blob/master/LICENSE)
 
 Python port of the queuer package - a queueing system based on PostgreSQL.
 
@@ -97,9 +98,15 @@ def example_task():
 That's easy, right? Adding a job is just as easy:
 
 ```python
-# Add a job to the queue
+# Add a job to the queue with positional arguments
 job = q.add_job(example_task, 5, "12")
 print(f"Job added: {job.rid}")
+
+# Add a job with keyword arguments (kwargs)
+job = q.add_job(example_task, paramKeyed1="test", debug=True)
+
+# Add a job with both positional and keyword arguments
+job = q.add_job(example_task, 5, "12", paramKeyed1="test", debug=True)
 ```
 
 In the initialisation of the queuer the existence of the necessary database tables is checked and if they don't exist they get created. The database is configured with these environment variables:
@@ -199,6 +206,72 @@ This method handles the registration of a task, making the worker able to pick u
 
 ---
 
+## add_job
+
+The `add_job` method adds a new job to the queue for execution. Jobs are units of work that will be processed by the queuer.
+
+```python
+def add_job(
+    self, 
+    task: Union[Callable, str], 
+    *parameters: Any,
+    **parameters_keyed: Any
+) -> Job
+
+def add_job_with_options(
+    self,
+    options: Optional[Options],
+    task: Union[Callable, str],
+    *parameters: Any,
+    **parameters_keyed: Any
+) -> Job
+```
+
+- `task`: A `Callable` or `str` representing the task to execute. If a callable, it must be registered with `add_task` first.
+- `options`: Optional `Options` for custom error handling or scheduling behavior. Only available in `add_job_with_options()`.
+- `*parameters`: Positional arguments to pass to the task function.
+- `**parameters_keyed`: Keyword arguments to pass to the task function (Python-specific feature).
+
+**Examples:**
+
+```python
+# Job with both positional and keyword arguments
+job = queuer.add_job(my_task, "arg1", paramKeyed1="test", paramKeyed2=1)
+
+# Job with custom options (requires add_job_with_options)
+options = Options(on_error=OnError(max_retries=5))
+job = queuer.add_job_with_options(options, my_task, "arg1", debug=True)
+```
+
+**Note:** Keyword arguments (`**parameters_keyed`) are stored separately in the database and enable Python functions to be called with named parameters. This is useful for optional parameters, default values, and improved code clarity. Go jobs continue to use only positional parameters.
+
+---
+
+## add_jobs
+
+The `add_jobs` method allows efficient batch insertion of multiple jobs at once.
+
+```python
+def add_jobs(self, batch_jobs: List[BatchJob]) -> List[UUID]
+```
+
+- `batch_jobs`: A list of `BatchJob` instances to insert.
+
+**Example:**
+
+```python
+from model.batch_job import BatchJob
+
+batch = [
+    BatchJob(task=my_task, parameters=[1, "a"]),
+    BatchJob(task=my_task, parameters=[2], parameters_keyed={"name": "b"}),
+    BatchJob(task=my_task, parameters_keyed={"id": 3, "name": "c"})
+]
+job_rids = queuer.add_jobs(batch)
+```
+
+---
+
 ## add_next_interval_func
 
 The `add_next_interval_func` method registers a custom function that determines the next execution time for scheduled jobs. This is useful for implementing complex scheduling logic beyond simple fixed intervals.
@@ -296,19 +369,20 @@ class Schedule:
 ### Job Management
 
 ```python
-# Add a single job
-job = queuer.add_job(my_task, param1, param2)
+# Add a job with both positional and keyword arguments
+job = queuer.add_job(my_task, param1, param2, paramKeyed1="test", paramKeyed2=1)
 
 # Add a job with custom options
 from model.options import Options, OnError
 options = Options(on_error=OnError(max_retries=5, timeout=60.0))
-job = queuer.add_job_with_options(options, my_task, param1)
+job = queuer.add_job_with_options(options, my_task, param1, paramKeyed1="test")
 
 # Add multiple jobs as a batch
 from model.batch_job import BatchJob
 batch = [
     BatchJob(task=my_task, parameters=[1, "a"]),
-    BatchJob(task=my_task, parameters=[2, "b"])
+    BatchJob(task=my_task, parameters=[2], parameters_keyed={"name": "b"}),
+    BatchJob(task=my_task, parameters_keyed={"id": 3, "name": "c"})
 ]
 queuer.add_jobs(batch)
 
@@ -335,6 +409,7 @@ worker_jobs = queuer.get_jobs_by_worker_rid(worker.rid)
 
 # ⭐ Features
 
+- **Keyword Arguments Support**: Python functions can use both positional arguments (`*args`) and keyword arguments (`**kwargs`), while maintaining compatibility with Go jobs that use only positional parameters.
 - **Async/Await Support**: Full asyncio integration with threading fallbacks.
 - **PostgreSQL NOTIFY/LISTEN**: Real-time job notifications without polling overhead.
 - **Batch Job Processing**: Insert job batches efficiently using PostgreSQL's `COPY FROM` feature.

{queuerpy-0.15.0 → queuerpy-0.17.0}/README.md

@@ -1,8 +1,9 @@
 # queuerPy
 
+[![PyPi](https://pythonico.leapcell.app/pypi/queuerPy.svg?style=shields&data=n,v,d)](https://pypi.org/project/queuerPy/)
 [![Python](https://img.shields.io/badge/Python-3.10+-blue.svg)](https://www.python.org/)
-[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/siherrmann/queuer/blob/master/LICENSE)
 ![Coverage](https://raw.githubusercontent.com/siherrmann/queuerPy/refs/heads/main/coverage-badge.svg)
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/siherrmann/queuer/blob/master/LICENSE)
 
 Python port of the queuer package - a queueing system based on PostgreSQL.
 
@@ -54,9 +55,15 @@ def example_task():
 That's easy, right? Adding a job is just as easy:
 
 ```python
-# Add a job to the queue
+# Add a job to the queue with positional arguments
 job = q.add_job(example_task, 5, "12")
 print(f"Job added: {job.rid}")
+
+# Add a job with keyword arguments (kwargs)
+job = q.add_job(example_task, paramKeyed1="test", debug=True)
+
+# Add a job with both positional and keyword arguments
+job = q.add_job(example_task, 5, "12", paramKeyed1="test", debug=True)
 ```
 
 In the initialisation of the queuer the existence of the necessary database tables is checked and if they don't exist they get created. The database is configured with these environment variables:
@@ -156,6 +163,72 @@ This method handles the registration of a task, making the worker able to pick u
 
 ---
 
+## add_job
+
+The `add_job` method adds a new job to the queue for execution. Jobs are units of work that will be processed by the queuer.
+
+```python
+def add_job(
+    self, 
+    task: Union[Callable, str], 
+    *parameters: Any,
+    **parameters_keyed: Any
+) -> Job
+
+def add_job_with_options(
+    self,
+    options: Optional[Options],
+    task: Union[Callable, str],
+    *parameters: Any,
+    **parameters_keyed: Any
+) -> Job
+```
+
+- `task`: A `Callable` or `str` representing the task to execute. If a callable, it must be registered with `add_task` first.
+- `options`: Optional `Options` for custom error handling or scheduling behavior. Only available in `add_job_with_options()`.
+- `*parameters`: Positional arguments to pass to the task function.
+- `**parameters_keyed`: Keyword arguments to pass to the task function (Python-specific feature).
+
+**Examples:**
+
+```python
+# Job with both positional and keyword arguments
+job = queuer.add_job(my_task, "arg1", paramKeyed1="test", paramKeyed2=1)
+
+# Job with custom options (requires add_job_with_options)
+options = Options(on_error=OnError(max_retries=5))
+job = queuer.add_job_with_options(options, my_task, "arg1", debug=True)
+```
+
+**Note:** Keyword arguments (`**parameters_keyed`) are stored separately in the database and enable Python functions to be called with named parameters. This is useful for optional parameters, default values, and improved code clarity. Go jobs continue to use only positional parameters.
+
+---
+
+## add_jobs
+
+The `add_jobs` method allows efficient batch insertion of multiple jobs at once.
+
+```python
+def add_jobs(self, batch_jobs: List[BatchJob]) -> List[UUID]
+```
+
+- `batch_jobs`: A list of `BatchJob` instances to insert.
+
+**Example:**
+
+```python
+from model.batch_job import BatchJob
+
+batch = [
+    BatchJob(task=my_task, parameters=[1, "a"]),
+    BatchJob(task=my_task, parameters=[2], parameters_keyed={"name": "b"}),
+    BatchJob(task=my_task, parameters_keyed={"id": 3, "name": "c"})
+]
+job_rids = queuer.add_jobs(batch)
+```
+
+---
+
 ## add_next_interval_func
 
 The `add_next_interval_func` method registers a custom function that determines the next execution time for scheduled jobs. This is useful for implementing complex scheduling logic beyond simple fixed intervals.
@@ -253,19 +326,20 @@ class Schedule:
 ### Job Management
 
 ```python
-# Add a single job
-job = queuer.add_job(my_task, param1, param2)
+# Add a job with both positional and keyword arguments
+job = queuer.add_job(my_task, param1, param2, paramKeyed1="test", paramKeyed2=1)
 
 # Add a job with custom options
 from model.options import Options, OnError
 options = Options(on_error=OnError(max_retries=5, timeout=60.0))
-job = queuer.add_job_with_options(options, my_task, param1)
+job = queuer.add_job_with_options(options, my_task, param1, paramKeyed1="test")
 
 # Add multiple jobs as a batch
 from model.batch_job import BatchJob
 batch = [
     BatchJob(task=my_task, parameters=[1, "a"]),
-    BatchJob(task=my_task, parameters=[2, "b"])
+    BatchJob(task=my_task, parameters=[2], parameters_keyed={"name": "b"}),
+    BatchJob(task=my_task, parameters_keyed={"id": 3, "name": "c"})
 ]
 queuer.add_jobs(batch)
 
@@ -292,6 +366,7 @@ worker_jobs = queuer.get_jobs_by_worker_rid(worker.rid)
 
 # ⭐ Features
 
+- **Keyword Arguments Support**: Python functions can use both positional arguments (`*args`) and keyword arguments (`**kwargs`), while maintaining compatibility with Go jobs that use only positional parameters.
 - **Async/Await Support**: Full asyncio integration with threading fallbacks.
 - **PostgreSQL NOTIFY/LISTEN**: Real-time job notifications without polling overhead.
 - **Batch Job Processing**: Insert job batches efficiently using PostgreSQL's `COPY FROM` feature.

{queuerpy-0.15.0 → queuerpy-0.17.0}/_version.py

@@ -1,6 +1,6 @@
 """Version information for queuerPy package."""
 
-__version__ = "0.15.0"
+__version__ = "0.17.0"
 __version_info__ = tuple(int(part) for part in __version__.split("."))
 
 # For compatibility

{queuerpy-0.15.0 → queuerpy-0.17.0}/database/db_job.py

@@ -95,11 +95,12 @@ class JobDBHandler:
 
         with self.db.instance.cursor(row_factory=dict_row) as cur:
             cur.execute(
-                "SELECT * FROM insert_job(%s, %s, %s, %s, %s, %s);",
+                "SELECT * FROM insert_job(%s, %s, %s, %s, %s, %s, %s);",
                 (
                     json.dumps(job.options.to_dict()) if job.options else None,
                     job.task_name,
                     json.dumps(job.parameters),
+                    json.dumps(job.parameters_keyed),
                     job.status,
                     job.scheduled_at,
                     job.schedule_count,
@@ -144,21 +145,28 @@ class JobDBHandler:
             raise ValueError("Database connection is not established")
 
         try:
-            batch_data: List[Tuple[str, str, str, Optional[datetime]]] = []
+            batch_data: List[Tuple[str, str, str, str, Optional[datetime]]] = []
             for job in jobs:
                 options_json = (
                     json.dumps(job.options.to_dict()) if job.options else "{}"
                 )
                 parameters_json = json.dumps(job.parameters)
+                parameters_keyed_json = json.dumps(job.parameters_keyed)
                 batch_data.append(
-                    (options_json, job.task_name, parameters_json, job.scheduled_at)
+                    (
+                        options_json,
+                        job.task_name,
+                        parameters_json,
+                        parameters_keyed_json,
+                        job.scheduled_at,
+                    )
                 )
 
             with self.db.instance.cursor(row_factory=dict_row) as cur:
                 cur.executemany(
                     """
-                    INSERT INTO job (options, task_name, parameters, scheduled_at) 
-                    VALUES (%s, %s, %s, %s)
+                    INSERT INTO job (options, task_name, parameters, parameters_keyed, scheduled_at) 
+                    VALUES (%s, %s, %s, %s, %s)
                     """,
                     batch_data,
                 )

{queuerpy-0.15.0 → queuerpy-0.17.0}/model/batch_job.py

@@ -3,8 +3,8 @@ BatchJob model for Python queuer implementation.
 Mirrors Go's model.BatchJob struct.
 """
 
-from typing import Any, List, Optional, Union, Callable
-from dataclasses import dataclass
+from typing import Any, Dict, List, Optional, Union, Callable
+from dataclasses import dataclass, field
 
 from .options import Options
 
@@ -16,5 +16,6 @@ class BatchJob:
     """
 
     task: Union[Callable[..., Any], str]
-    parameters: List[Any]
+    parameters: List[Any] = field(default_factory=list)
+    parameters_keyed: Dict[str, Any] = field(default_factory=dict)
     options: Optional[Options] = None

{queuerpy-0.15.0 → queuerpy-0.17.0}/model/job.py

@@ -40,7 +40,8 @@ class Job:
 
     # Job definition
     task_name: str = ""
-    parameters: List[Any] = field(default_factory=lambda: [])
+    parameters: List[Any] = field(default_factory=list)
+    parameters_keyed: Dict[str, Any] = field(default_factory=dict)
     options: Optional[Options] = None
 
     # Job state
@@ -65,6 +66,7 @@ class Job:
             "worker_rid": str(self.worker_rid),
             "task_name": self.task_name,
             "parameters": self.parameters,
+            "parameters_keyed": self.parameters_keyed,
             "options": self.options.to_dict() if self.options else None,
             "status": self.status,
             "scheduled_at": (
@@ -110,6 +112,7 @@ class Job:
             job.worker_rid = UUID(data["worker_rid"])
         job.task_name = data.get("task_name", "")
         job.parameters = data.get("parameters", [])
+        job.parameters_keyed = data.get("parameters_keyed", {})
         if data.get("options"):
             job.options = Options.from_dict(data["options"])
         job.status = data.get("status", JobStatus.QUEUED)
@@ -173,6 +176,16 @@ class Job:
                 else parameters_value
             )
 
+        parameters_keyed_value = row.get(
+            "output_parameters_keyed", row.get("parameters_keyed")
+        )
+        if parameters_keyed_value:
+            job.parameters_keyed = (
+                json.loads(parameters_keyed_value)
+                if isinstance(parameters_keyed_value, str)
+                else parameters_keyed_value
+            )
+
         results_value = row.get("output_results", row.get("results"))
         if results_value is not None:
             # JSONB columns return Python objects directly, no JSON parsing needed
@@ -194,6 +207,7 @@ def new_job(
     task: Union[Callable[..., Any], str],
     options: Optional[Options] = None,
     *parameters: Any,
+    **parameters_keyed: Any,
 ) -> Job:
     """
     Create a new job from a task function or task name.
@@ -237,6 +251,7 @@ def new_job(
     job: Job = Job()
     job.task_name = task_name
     job.parameters = list(parameters)
+    job.parameters_keyed = parameters_keyed
     job.options = options
     job.status = status
     job.scheduled_at = scheduled_at

{queuerpy-0.15.0 → queuerpy-0.17.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "queuerPy"
-version = "0.15.0"
+version = "0.17.0"
 description = "A Python implementation of the queuer system - a job queuing and processing system with PostgreSQL backend"
 readme = "README.md"
 license = "Apache-2.0"
@@ -92,7 +92,7 @@ pythonVersion = "3.8"
 pythonPlatform = "All"
 
 [tool.mypy]
-python_version = "0.15.0"
+python_version = "0.17.0"
 warn_return_any = true
 warn_unused_configs = true
 disallow_untyped_defs = true

{queuerpy-0.15.0 → queuerpy-0.17.0/queuerPy.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: queuerPy
-Version: 0.15.0
+Version: 0.17.0
 Summary: A Python implementation of the queuer system - a job queuing and processing system with PostgreSQL backend
 Author-email: Simon Herrmann <siherrmann@users.noreply.github.com>
 Maintainer-email: Simon Herrmann <siherrmann@users.noreply.github.com>
@@ -43,9 +43,10 @@ Dynamic: license-file
 
 # queuerPy
 
+[![PyPi](https://pythonico.leapcell.app/pypi/queuerPy.svg?style=shields&data=n,v,d)](https://pypi.org/project/queuerPy/)
 [![Python](https://img.shields.io/badge/Python-3.10+-blue.svg)](https://www.python.org/)
-[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/siherrmann/queuer/blob/master/LICENSE)
 ![Coverage](https://raw.githubusercontent.com/siherrmann/queuerPy/refs/heads/main/coverage-badge.svg)
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/siherrmann/queuer/blob/master/LICENSE)
 
 Python port of the queuer package - a queueing system based on PostgreSQL.
 
@@ -97,9 +98,15 @@ def example_task():
 That's easy, right? Adding a job is just as easy:
 
 ```python
-# Add a job to the queue
+# Add a job to the queue with positional arguments
 job = q.add_job(example_task, 5, "12")
 print(f"Job added: {job.rid}")
+
+# Add a job with keyword arguments (kwargs)
+job = q.add_job(example_task, paramKeyed1="test", debug=True)
+
+# Add a job with both positional and keyword arguments
+job = q.add_job(example_task, 5, "12", paramKeyed1="test", debug=True)
 ```
 
 In the initialisation of the queuer the existence of the necessary database tables is checked and if they don't exist they get created. The database is configured with these environment variables:
@@ -199,6 +206,72 @@ This method handles the registration of a task, making the worker able to pick u
 
 ---
 
+## add_job
+
+The `add_job` method adds a new job to the queue for execution. Jobs are units of work that will be processed by the queuer.
+
+```python
+def add_job(
+    self, 
+    task: Union[Callable, str], 
+    *parameters: Any,
+    **parameters_keyed: Any
+) -> Job
+
+def add_job_with_options(
+    self,
+    options: Optional[Options],
+    task: Union[Callable, str],
+    *parameters: Any,
+    **parameters_keyed: Any
+) -> Job
+```
+
+- `task`: A `Callable` or `str` representing the task to execute. If a callable, it must be registered with `add_task` first.
+- `options`: Optional `Options` for custom error handling or scheduling behavior. Only available in `add_job_with_options()`.
+- `*parameters`: Positional arguments to pass to the task function.
+- `**parameters_keyed`: Keyword arguments to pass to the task function (Python-specific feature).
+
+**Examples:**
+
+```python
+# Job with both positional and keyword arguments
+job = queuer.add_job(my_task, "arg1", paramKeyed1="test", paramKeyed2=1)
+
+# Job with custom options (requires add_job_with_options)
+options = Options(on_error=OnError(max_retries=5))
+job = queuer.add_job_with_options(options, my_task, "arg1", debug=True)
+```
+
+**Note:** Keyword arguments (`**parameters_keyed`) are stored separately in the database and enable Python functions to be called with named parameters. This is useful for optional parameters, default values, and improved code clarity. Go jobs continue to use only positional parameters.
+
+---
+
+## add_jobs
+
+The `add_jobs` method allows efficient batch insertion of multiple jobs at once.
+
+```python
+def add_jobs(self, batch_jobs: List[BatchJob]) -> List[UUID]
+```
+
+- `batch_jobs`: A list of `BatchJob` instances to insert.
+
+**Example:**
+
+```python
+from model.batch_job import BatchJob
+
+batch = [
+    BatchJob(task=my_task, parameters=[1, "a"]),
+    BatchJob(task=my_task, parameters=[2], parameters_keyed={"name": "b"}),
+    BatchJob(task=my_task, parameters_keyed={"id": 3, "name": "c"})
+]
+job_rids = queuer.add_jobs(batch)
+```
+
+---
+
 ## add_next_interval_func
 
 The `add_next_interval_func` method registers a custom function that determines the next execution time for scheduled jobs. This is useful for implementing complex scheduling logic beyond simple fixed intervals.
@@ -296,19 +369,20 @@ class Schedule:
 ### Job Management
 
 ```python
-# Add a single job
-job = queuer.add_job(my_task, param1, param2)
+# Add a job with both positional and keyword arguments
+job = queuer.add_job(my_task, param1, param2, paramKeyed1="test", paramKeyed2=1)
 
 # Add a job with custom options
 from model.options import Options, OnError
 options = Options(on_error=OnError(max_retries=5, timeout=60.0))
-job = queuer.add_job_with_options(options, my_task, param1)
+job = queuer.add_job_with_options(options, my_task, param1, paramKeyed1="test")
 
 # Add multiple jobs as a batch
 from model.batch_job import BatchJob
 batch = [
     BatchJob(task=my_task, parameters=[1, "a"]),
-    BatchJob(task=my_task, parameters=[2, "b"])
+    BatchJob(task=my_task, parameters=[2], parameters_keyed={"name": "b"}),
+    BatchJob(task=my_task, parameters_keyed={"id": 3, "name": "c"})
 ]
 queuer.add_jobs(batch)
 
@@ -335,6 +409,7 @@ worker_jobs = queuer.get_jobs_by_worker_rid(worker.rid)
 
 # ⭐ Features
 
+- **Keyword Arguments Support**: Python functions can use both positional arguments (`*args`) and keyword arguments (`**kwargs`), while maintaining compatibility with Go jobs that use only positional parameters.
 - **Async/Await Support**: Full asyncio integration with threading fallbacks.
 - **PostgreSQL NOTIFY/LISTEN**: Real-time job notifications without polling overhead.
 - **Batch Job Processing**: Insert job batches efficiently using PostgreSQL's `COPY FROM` feature.

{queuerpy-0.15.0 → queuerpy-0.17.0}/queuer_job.py

@@ -34,17 +34,23 @@ class QueuerJobMixin(QueuerGlobalMixin):
     def __init__(self):
         super().__init__()
 
-    def add_job(self, task: Union[Callable[..., Any], str], *parameters: Any) -> Job:
+    def add_job(
+        self,
+        task: Union[Callable[..., Any], str],
+        *parameters: Any,
+        **parameters_keyed: Any,
+    ) -> Job:
         """
         Add a job to the queue with the given task and parameters.
 
         :param task: Either a function or a string with the task name
-        :param parameters: Parameters to pass to the task
+        :param parameters: Positional parameters to pass to the task
+        :param parameters_keyed: Keyword parameters to pass to the task
         :returns: The created job
         :raises Exception: If something goes wrong
         """
         options: Optional[Options] = self._merge_options(None)
-        job: Job = self._add_job(task, options, *parameters)
+        job: Job = self._add_job(task, options, *parameters, **parameters_keyed)
 
         logger.info(f"Job added: {job.rid}")
 
@@ -55,6 +61,7 @@ class QueuerJobMixin(QueuerGlobalMixin):
         options: Optional[Options],
         task: Union[Callable[..., Any], str],
         *parameters: Any,
+        **parameters_keyed: Any,
     ) -> Job:
         """Add a new job with specific options."""
         # Merge default options with provided options
@@ -62,7 +69,9 @@ class QueuerJobMixin(QueuerGlobalMixin):
 
         try:
             # Create new job
-            new_job: Job = create_job(task, options_merged, *parameters)
+            new_job: Job = create_job(
+                task, options_merged, *parameters, **parameters_keyed
+            )
             job: Job = self.db_job.insert_job(new_job)
 
             logger.info(f"Job with options added: {job.rid}")
@@ -82,7 +91,9 @@ class QueuerJobMixin(QueuerGlobalMixin):
         for batch_job in batch_jobs:
             options: Optional[Options] = self._merge_options(batch_job.options)
             task_name: str = get_task_name_from_interface(batch_job.task)
-            job: Job = create_job(task_name, options, *batch_job.parameters)
+            job: Job = create_job(
+                task_name, options, *batch_job.parameters, **batch_job.parameters_keyed
+            )
             jobs.append(job)
 
         self.db_job.batch_insert_jobs(jobs)
@@ -232,7 +243,9 @@ class QueuerJobMixin(QueuerGlobalMixin):
             raise Exception(f"Job not found in archive: {job_rid}")
 
         # Readd the job to the queue
-        new_job = self.add_job_with_options(job.options, job.task_name, *job.parameters)
+        new_job = self.add_job_with_options(
+            job.options, job.task_name, *job.parameters, **job.parameters_keyed
+        )
         logger.info(f"Job readded: {new_job.rid}")
         return new_job
 
@@ -317,18 +330,20 @@ class QueuerJobMixin(QueuerGlobalMixin):
         task: Union[Callable[..., Any], str],
         options: Optional["Options"],
         *parameters: Any,
+        **parameters_keyed: Any,
     ) -> Job:
         """
         Add a job to the queue with all necessary parameters.
 
         :param task: Either a function or a string with the task name
         :param options: Job-specific options (can be None)
-        :param parameters: Parameters to pass to the task
+        :param parameters: Positional parameters to pass to the task
+        :param parameters_keyed: Keyword parameters to pass to the task
         :returns: The created job
         :raises Exception: If something goes wrong
         """
         try:
-            new_job = create_job(task, options, *parameters)
+            new_job = create_job(task, options, *parameters, **parameters_keyed)
             job = self.db_job.insert_job(new_job)
 
             return job
@@ -405,7 +420,8 @@ class QueuerJobMixin(QueuerGlobalMixin):
 
         try:
             parameters = getattr(job, "parameters", [])
-            runner = Runner(task, *parameters)
+            parameters_keyed = getattr(job, "parameters_keyed", {})
+            runner = Runner(task, *parameters, **parameters_keyed)
 
             logger.info(f"Created runner for job {job.rid}")
 

{queuerpy-0.15.0 → queuerpy-0.17.0}/sql/job.sql

@@ -15,6 +15,9 @@ BEGIN
         options JSONB DEFAULT '{}',
         task_name VARCHAR(100) DEFAULT '',
         parameters JSONB DEFAULT '[]',
+        parameters_encrypted BYTEA DEFAULT '',
+        parameters_keyed JSONB DEFAULT '{}',
+        parameters_keyed_encrypted BYTEA DEFAULT '',
         status VARCHAR(50) DEFAULT 'QUEUED',
         scheduled_at TIMESTAMP DEFAULT NULL,
         started_at TIMESTAMP DEFAULT NULL,
@@ -62,6 +65,7 @@ CREATE OR REPLACE FUNCTION insert_job(
     input_options JSONB,
     input_task_name VARCHAR(100),
     input_parameters JSONB,
+    input_parameters_keyed JSONB,
     input_status VARCHAR(50),
     input_scheduled_at TIMESTAMP,
     input_schedule_count INT
@@ -74,6 +78,7 @@ RETURNS TABLE (
     output_options JSONB,
     output_task_name VARCHAR(100),
     output_parameters JSONB,
+    output_parameters_keyed JSONB,
     output_status VARCHAR(50),
     output_scheduled_at TIMESTAMP,
     output_schedule_count INT,
@@ -84,8 +89,8 @@ RETURNS TABLE (
 AS $$
 BEGIN
     RETURN QUERY
-    INSERT INTO job (options, task_name, parameters, status, scheduled_at, schedule_count)
-    VALUES (input_options, input_task_name, input_parameters, input_status, input_scheduled_at, input_schedule_count)
+    INSERT INTO job (options, task_name, parameters, parameters_keyed, status, scheduled_at, schedule_count)
+    VALUES (input_options, input_task_name, input_parameters, input_parameters_keyed, input_status, input_scheduled_at, input_schedule_count)
     RETURNING
         job.id,
         job.rid,
@@ -94,6 +99,7 @@ BEGIN
         job.options,
         job.task_name,
         job.parameters,
+        job.parameters_keyed,
         job.status,
         job.scheduled_at,
         job.schedule_count,
@@ -112,6 +118,7 @@ RETURNS TABLE (
     output_options JSONB,
     output_task_name VARCHAR(100),
     output_parameters JSONB,
+    output_parameters_keyed JSONB,
     output_status VARCHAR(50),
     output_scheduled_at TIMESTAMP,
     output_started_at TIMESTAMP,
@@ -182,6 +189,7 @@ BEGIN
         job.options,
         job.task_name,
         job.parameters,
+        job.parameters_keyed,
         job.status,
         job.scheduled_at,
         job.started_at,
@@ -206,6 +214,7 @@ RETURNS TABLE (
     output_options JSONB,
     output_task_name VARCHAR(100),
     output_parameters JSONB,
+    output_parameters_keyed JSONB,
     output_status VARCHAR(50),
     output_scheduled_at TIMESTAMP,
     output_started_at TIMESTAMP,
@@ -229,6 +238,9 @@ BEGIN
             options,
             task_name,
             parameters,
+            parameters_encrypted,
+            parameters_keyed,
+            parameters_keyed_encrypted,
             scheduled_at,
             started_at,
             schedule_count,
@@ -244,6 +256,9 @@ BEGIN
         options,
         task_name,
         parameters,
+        parameters_encrypted,
+        parameters_keyed,
+        parameters_keyed_encrypted,
         status,
         scheduled_at,
         started_at,
@@ -262,6 +277,9 @@ BEGIN
         jobs_old.options,
         jobs_old.task_name,
         jobs_old.parameters,
+        jobs_old.parameters_encrypted,
+        jobs_old.parameters_keyed,
+        jobs_old.parameters_keyed_encrypted,
         input_status,
         jobs_old.scheduled_at,
         jobs_old.started_at,
@@ -280,6 +298,7 @@ BEGIN
         options,
         task_name,
         parameters,
+        parameters_keyed,
         status,
         scheduled_at,
         started_at,
@@ -307,6 +326,7 @@ RETURNS TABLE (
     output_options JSONB,
     output_task_name VARCHAR(100),
     output_parameters JSONB,
+    output_parameters_keyed JSONB,
     output_status VARCHAR(50),
     output_scheduled_at TIMESTAMP,
     output_started_at TIMESTAMP,
@@ -330,6 +350,9 @@ BEGIN
             options,
             task_name,
             parameters,
+            parameters_encrypted,
+            parameters_keyed,
+            parameters_keyed_encrypted,
             scheduled_at,
             started_at,
             schedule_count,
@@ -345,6 +368,9 @@ BEGIN
         options,
         task_name,
         parameters,
+        parameters_encrypted,
+        parameters_keyed,
+        parameters_keyed_encrypted,
         status,
         scheduled_at,
         started_at,
@@ -363,6 +389,9 @@ BEGIN
         jobs_old.options,
         jobs_old.task_name,
         jobs_old.parameters,
+        jobs_old.parameters_encrypted,
+        jobs_old.parameters_keyed,
+        jobs_old.parameters_keyed_encrypted,
         input_status,
         jobs_old.scheduled_at,
         jobs_old.started_at,
@@ -381,6 +410,7 @@ BEGIN
         options,
         task_name,
         parameters,
+        parameters_keyed,
         status,
         scheduled_at,
         started_at,
@@ -419,13 +449,6 @@ BEGIN
 END;
 $$ LANGUAGE plpgsql;
 
-CREATE OR REPLACE FUNCTION delete_job(input_rid UUID)
-RETURNS VOID AS $$
-BEGIN
-    DELETE FROM job WHERE rid = input_rid;
-END;
-$$ LANGUAGE plpgsql;
-
 CREATE OR REPLACE FUNCTION select_job(input_encryption_key TEXT, input_rid UUID)
 RETURNS TABLE (
     output_id BIGINT,
@@ -435,6 +458,7 @@ RETURNS TABLE (
     output_options JSONB,
     output_task_name VARCHAR(100),
     output_parameters JSONB,
+    output_parameters_keyed JSONB,
     output_status VARCHAR(50),
     output_scheduled_at TIMESTAMP,
     output_started_at TIMESTAMP,
@@ -454,7 +478,14 @@ BEGIN
         job.worker_rid,
         job.options,
         job.task_name,
-        job.parameters,
+        CASE
+            WHEN octet_length(job.parameters_encrypted) > 0 THEN pgp_sym_decrypt(job.parameters_encrypted, input_encryption_key::text)::jsonb
+            ELSE job.parameters
+        END AS parameters,
+        CASE
+            WHEN octet_length(job.parameters_keyed_encrypted) > 0 THEN pgp_sym_decrypt(job.parameters_keyed_encrypted, input_encryption_key::text)::jsonb
+            ELSE job.parameters_keyed
+        END AS parameters_keyed,
         job.status,
         job.scheduled_at,
         job.started_at,
@@ -481,6 +512,7 @@ RETURNS TABLE (
     output_options JSONB,
     output_task_name VARCHAR(100),
     output_parameters JSONB,
+    output_parameters_keyed JSONB,
     output_status VARCHAR(50),
     output_scheduled_at TIMESTAMP,
     output_started_at TIMESTAMP,
@@ -500,7 +532,14 @@ BEGIN
         job.worker_rid,
         job.options,
         job.task_name,
-        job.parameters,
+        CASE
+            WHEN octet_length(job.parameters_encrypted) > 0 THEN pgp_sym_decrypt(job.parameters_encrypted, input_encryption_key::text)::jsonb
+            ELSE job.parameters
+        END AS parameters,
+        CASE
+            WHEN octet_length(job.parameters_keyed_encrypted) > 0 THEN pgp_sym_decrypt(job.parameters_keyed_encrypted, input_encryption_key::text)::jsonb
+            ELSE job.parameters_keyed
+        END AS parameters_keyed,
         job.status,
         job.scheduled_at,
         job.started_at,
@@ -538,6 +577,7 @@ RETURNS TABLE (
     output_options JSONB,
     output_task_name VARCHAR(100),
     output_parameters JSONB,
+    output_parameters_keyed JSONB,
     output_status VARCHAR(50),
     output_scheduled_at TIMESTAMP,
     output_started_at TIMESTAMP,
@@ -557,7 +597,14 @@ BEGIN
         job.worker_rid,
         job.options,
         job.task_name,
-        job.parameters,
+        CASE
+            WHEN octet_length(job.parameters_encrypted) > 0 THEN pgp_sym_decrypt(job.parameters_encrypted, input_encryption_key::text)::jsonb
+            ELSE job.parameters
+        END AS parameters,
+        CASE
+            WHEN octet_length(job.parameters_keyed_encrypted) > 0 THEN pgp_sym_decrypt(job.parameters_keyed_encrypted, input_encryption_key::text)::jsonb
+            ELSE job.parameters_keyed
+        END AS parameters_keyed,
         job.status,
         job.scheduled_at,
         job.started_at,
@@ -596,6 +643,7 @@ RETURNS TABLE (
     output_options JSONB,
     output_task_name VARCHAR(100),
     output_parameters JSONB,
+    output_parameters_keyed JSONB,
     output_status VARCHAR(50),
     output_scheduled_at TIMESTAMP,
     output_started_at TIMESTAMP,
@@ -615,7 +663,14 @@ BEGIN
         job.worker_rid,
         job.options,
         job.task_name,
-        job.parameters,
+        CASE
+            WHEN octet_length(job.parameters_encrypted) > 0 THEN pgp_sym_decrypt(job.parameters_encrypted, input_encryption_key::text)::jsonb
+            ELSE job.parameters
+        END AS parameters,
+        CASE
+            WHEN octet_length(job.parameters_keyed_encrypted) > 0 THEN pgp_sym_decrypt(job.parameters_keyed_encrypted, input_encryption_key::text)::jsonb
+            ELSE job.parameters_keyed
+        END AS parameters_keyed,
         job.status,
         job.scheduled_at,
         job.started_at,
@@ -657,6 +712,13 @@ BEGIN
 END;
 $$ LANGUAGE plpgsql;
 
+CREATE OR REPLACE FUNCTION delete_job(input_rid UUID)
+RETURNS VOID AS $$
+BEGIN
+    DELETE FROM job_archive WHERE rid = input_rid;
+END;
+$$ LANGUAGE plpgsql;
+
 CREATE OR REPLACE FUNCTION select_job_from_archive(input_encryption_key TEXT, input_rid UUID)
 RETURNS TABLE (
     output_id BIGINT,
@@ -666,6 +728,7 @@ RETURNS TABLE (
     output_options JSONB,
     output_task_name VARCHAR(100),
     output_parameters JSONB,
+    output_parameters_keyed JSONB,
     output_status VARCHAR(50),
     output_scheduled_at TIMESTAMP,
     output_started_at TIMESTAMP,
@@ -685,7 +748,14 @@ BEGIN
         job_archive.worker_rid,
         job_archive.options,
         job_archive.task_name,
-        job_archive.parameters,
+        CASE
+            WHEN octet_length(job_archive.parameters_encrypted) > 0 THEN pgp_sym_decrypt(job_archive.parameters_encrypted, input_encryption_key::text)::jsonb
+            ELSE job_archive.parameters
+        END AS parameters,
+        CASE
+            WHEN octet_length(job_archive.parameters_keyed_encrypted) > 0 THEN pgp_sym_decrypt(job_archive.parameters_keyed_encrypted, input_encryption_key::text)::jsonb
+            ELSE job_archive.parameters_keyed
+        END AS parameters_keyed,
         job_archive.status,
         job_archive.scheduled_at,
         job_archive.started_at,
@@ -712,6 +782,7 @@ RETURNS TABLE (
     output_options JSONB,
     output_task_name VARCHAR(100),
     output_parameters JSONB,
+    output_parameters_keyed JSONB,
     output_status VARCHAR(50),
     output_scheduled_at TIMESTAMP,
     output_started_at TIMESTAMP,
@@ -731,7 +802,14 @@ BEGIN
         job_archive.worker_rid,
         job_archive.options,
         job_archive.task_name,
-        job_archive.parameters,
+        CASE
+            WHEN octet_length(job_archive.parameters_encrypted) > 0 THEN pgp_sym_decrypt(job_archive.parameters_encrypted, input_encryption_key::text)::jsonb
+            ELSE job_archive.parameters
+        END AS parameters,
+        CASE
+            WHEN octet_length(job_archive.parameters_keyed_encrypted) > 0 THEN pgp_sym_decrypt(job_archive.parameters_keyed_encrypted, input_encryption_key::text)::jsonb
+            ELSE job_archive.parameters_keyed
+        END AS parameters_keyed,
         job_archive.status,
         job_archive.scheduled_at,
         job_archive.started_at,
@@ -769,6 +847,7 @@ RETURNS TABLE (
     output_options JSONB,
     output_task_name VARCHAR(100),
     output_parameters JSONB,
+    output_parameters_keyed JSONB,
     output_status VARCHAR(50),
     output_scheduled_at TIMESTAMP,
     output_started_at TIMESTAMP,
@@ -788,7 +867,14 @@ BEGIN
         job_archive.worker_rid,
         job_archive.options,
         job_archive.task_name,
-        job_archive.parameters,
+        CASE
+            WHEN octet_length(job_archive.parameters_encrypted) > 0 THEN pgp_sym_decrypt(job_archive.parameters_encrypted, input_encryption_key::text)::jsonb
+            ELSE job_archive.parameters
+        END AS parameters,
+        CASE
+            WHEN octet_length(job_archive.parameters_keyed_encrypted) > 0 THEN pgp_sym_decrypt(job_archive.parameters_keyed_encrypted, input_encryption_key::text)::jsonb
+            ELSE job_archive.parameters_keyed
+        END AS parameters_keyed,
         job_archive.status,
         job_archive.scheduled_at,
         job_archive.started_at,

{queuerpy-0.15.0 → queuerpy-0.17.0}/sql/worker.sql

@@ -138,6 +138,23 @@ BEGIN
 END;
 $$ LANGUAGE plpgsql;
 
+CREATE OR REPLACE FUNCTION delete_stale_workers(
+    input_cutoff_time TIMESTAMP
+)
+RETURNS INT
+AS $$
+DECLARE
+    affected_rows INT;
+BEGIN
+    DELETE FROM worker
+    WHERE updated_at < input_cutoff_time
+      AND status = 'STOPPED';
+    
+    GET DIAGNOSTICS affected_rows = ROW_COUNT;
+    RETURN affected_rows;
+END;
+$$ LANGUAGE plpgsql;
+
 CREATE OR REPLACE FUNCTION select_worker(input_rid UUID)
 RETURNS TABLE (
     output_id BIGINT,