queuerPy 0.16.0__tar.gz → 0.18.0__tar.gz

{queuerpy-0.16.0/queuerPy.egg-info → queuerpy-0.18.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: queuerPy
-Version: 0.16.0
+Version: 0.18.0
 Summary: A Python implementation of the queuer system - a job queuing and processing system with PostgreSQL backend
 Author-email: Simon Herrmann <siherrmann@users.noreply.github.com>
 Maintainer-email: Simon Herrmann <siherrmann@users.noreply.github.com>
@@ -43,9 +43,10 @@ Dynamic: license-file
 
 # queuerPy
 
+[![PyPi](https://pythonico.leapcell.app/pypi/queuerPy.svg?style=shields&data=n,v,d)](https://pypi.org/project/queuerPy/)
 [![Python](https://img.shields.io/badge/Python-3.10+-blue.svg)](https://www.python.org/)
-[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/siherrmann/queuer/blob/master/LICENSE)
 ![Coverage](https://raw.githubusercontent.com/siherrmann/queuerPy/refs/heads/main/coverage-badge.svg)
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/siherrmann/queuer/blob/master/LICENSE)
 
 Python port of the queuer package - a queueing system based on PostgreSQL.
 
@@ -97,9 +98,15 @@ def example_task():
 That's easy, right? Adding a job is just as easy:
 
 ```python
-# Add a job to the queue
+# Add a job to the queue with positional arguments
 job = q.add_job(example_task, 5, "12")
 print(f"Job added: {job.rid}")
+
+# Add a job with keyword arguments (kwargs)
+job = q.add_job(example_task, paramKeyed1="test", debug=True)
+
+# Add a job with both positional and keyword arguments
+job = q.add_job(example_task, 5, "12", paramKeyed1="test", debug=True)
 ```
 
 In the initialisation of the queuer the existence of the necessary database tables is checked and if they don't exist they get created. The database is configured with these environment variables:
@@ -199,6 +206,72 @@ This method handles the registration of a task, making the worker able to pick u
 
 ---
 
+## add_job
+
+The `add_job` method adds a new job to the queue for execution. Jobs are units of work that will be processed by the queuer.
+
+```python
+def add_job(
+    self, 
+    task: Union[Callable, str], 
+    *parameters: Any,
+    **parameters_keyed: Any
+) -> Job
+
+def add_job_with_options(
+    self,
+    options: Optional[Options],
+    task: Union[Callable, str],
+    *parameters: Any,
+    **parameters_keyed: Any
+) -> Job
+```
+
+- `task`: A `Callable` or `str` representing the task to execute. If a callable, it must be registered with `add_task` first.
+- `options`: Optional `Options` for custom error handling or scheduling behavior. Only available in `add_job_with_options()`.
+- `*parameters`: Positional arguments to pass to the task function.
+- `**parameters_keyed`: Keyword arguments to pass to the task function (Python-specific feature).
+
+**Examples:**
+
+```python
+# Job with both positional and keyword arguments
+job = queuer.add_job(my_task, "arg1", paramKeyed1="test", paramKeyed2=1)
+
+# Job with custom options (requires add_job_with_options)
+options = Options(on_error=OnError(max_retries=5))
+job = queuer.add_job_with_options(options, my_task, "arg1", debug=True)
+```
+
+**Note:** Keyword arguments (`**parameters_keyed`) are stored separately in the database and enable Python functions to be called with named parameters. This is useful for optional parameters, default values, and improved code clarity. Go jobs continue to use only positional parameters.
+
+---
+
+## add_jobs
+
+The `add_jobs` method allows efficient batch insertion of multiple jobs at once.
+
+```python
+def add_jobs(self, batch_jobs: List[BatchJob]) -> List[UUID]
+```
+
+- `batch_jobs`: A list of `BatchJob` instances to insert.
+
+**Example:**
+
+```python
+from model.batch_job import BatchJob
+
+batch = [
+    BatchJob(task=my_task, parameters=[1, "a"]),
+    BatchJob(task=my_task, parameters=[2], parameters_keyed={"name": "b"}),
+    BatchJob(task=my_task, parameters_keyed={"id": 3, "name": "c"})
+]
+job_rids = queuer.add_jobs(batch)
+```
+
+---
+
 ## add_next_interval_func
 
 The `add_next_interval_func` method registers a custom function that determines the next execution time for scheduled jobs. This is useful for implementing complex scheduling logic beyond simple fixed intervals.
@@ -296,19 +369,20 @@ class Schedule:
 ### Job Management
 
 ```python
-# Add a single job
-job = queuer.add_job(my_task, param1, param2)
+# Add a job with both positional and keyword arguments
+job = queuer.add_job(my_task, param1, param2, paramKeyed1="test", paramKeyed2=1)
 
 # Add a job with custom options
 from model.options import Options, OnError
 options = Options(on_error=OnError(max_retries=5, timeout=60.0))
-job = queuer.add_job_with_options(options, my_task, param1)
+job = queuer.add_job_with_options(options, my_task, param1, paramKeyed1="test")
 
 # Add multiple jobs as a batch
 from model.batch_job import BatchJob
 batch = [
     BatchJob(task=my_task, parameters=[1, "a"]),
-    BatchJob(task=my_task, parameters=[2, "b"])
+    BatchJob(task=my_task, parameters=[2], parameters_keyed={"name": "b"}),
+    BatchJob(task=my_task, parameters_keyed={"id": 3, "name": "c"})
 ]
 queuer.add_jobs(batch)
 
@@ -335,6 +409,7 @@ worker_jobs = queuer.get_jobs_by_worker_rid(worker.rid)
 
 # ⭐ Features
 
+- **Keyword Arguments Support**: Python functions can use both positional arguments (`*args`) and keyword arguments (`**kwargs`), while maintaining compatibility with Go jobs that use only positional parameters.
 - **Async/Await Support**: Full asyncio integration with threading fallbacks.
 - **PostgreSQL NOTIFY/LISTEN**: Real-time job notifications without polling overhead.
 - **Batch Job Processing**: Insert job batches efficiently using PostgreSQL's `COPY FROM` feature.

{queuerpy-0.16.0 → queuerpy-0.18.0}/README.md

@@ -1,8 +1,9 @@
 # queuerPy
 
+[![PyPi](https://pythonico.leapcell.app/pypi/queuerPy.svg?style=shields&data=n,v,d)](https://pypi.org/project/queuerPy/)
 [![Python](https://img.shields.io/badge/Python-3.10+-blue.svg)](https://www.python.org/)
-[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/siherrmann/queuer/blob/master/LICENSE)
 ![Coverage](https://raw.githubusercontent.com/siherrmann/queuerPy/refs/heads/main/coverage-badge.svg)
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/siherrmann/queuer/blob/master/LICENSE)
 
 Python port of the queuer package - a queueing system based on PostgreSQL.
 
@@ -54,9 +55,15 @@ def example_task():
 That's easy, right? Adding a job is just as easy:
 
 ```python
-# Add a job to the queue
+# Add a job to the queue with positional arguments
 job = q.add_job(example_task, 5, "12")
 print(f"Job added: {job.rid}")
+
+# Add a job with keyword arguments (kwargs)
+job = q.add_job(example_task, paramKeyed1="test", debug=True)
+
+# Add a job with both positional and keyword arguments
+job = q.add_job(example_task, 5, "12", paramKeyed1="test", debug=True)
 ```
 
 In the initialisation of the queuer the existence of the necessary database tables is checked and if they don't exist they get created. The database is configured with these environment variables:
@@ -156,6 +163,72 @@ This method handles the registration of a task, making the worker able to pick u
 
 ---
 
+## add_job
+
+The `add_job` method adds a new job to the queue for execution. Jobs are units of work that will be processed by the queuer.
+
+```python
+def add_job(
+    self, 
+    task: Union[Callable, str], 
+    *parameters: Any,
+    **parameters_keyed: Any
+) -> Job
+
+def add_job_with_options(
+    self,
+    options: Optional[Options],
+    task: Union[Callable, str],
+    *parameters: Any,
+    **parameters_keyed: Any
+) -> Job
+```
+
+- `task`: A `Callable` or `str` representing the task to execute. If a callable, it must be registered with `add_task` first.
+- `options`: Optional `Options` for custom error handling or scheduling behavior. Only available in `add_job_with_options()`.
+- `*parameters`: Positional arguments to pass to the task function.
+- `**parameters_keyed`: Keyword arguments to pass to the task function (Python-specific feature).
+
+**Examples:**
+
+```python
+# Job with both positional and keyword arguments
+job = queuer.add_job(my_task, "arg1", paramKeyed1="test", paramKeyed2=1)
+
+# Job with custom options (requires add_job_with_options)
+options = Options(on_error=OnError(max_retries=5))
+job = queuer.add_job_with_options(options, my_task, "arg1", debug=True)
+```
+
+**Note:** Keyword arguments (`**parameters_keyed`) are stored separately in the database and enable Python functions to be called with named parameters. This is useful for optional parameters, default values, and improved code clarity. Go jobs continue to use only positional parameters.
+
+---
+
+## add_jobs
+
+The `add_jobs` method allows efficient batch insertion of multiple jobs at once.
+
+```python
+def add_jobs(self, batch_jobs: List[BatchJob]) -> List[UUID]
+```
+
+- `batch_jobs`: A list of `BatchJob` instances to insert.
+
+**Example:**
+
+```python
+from model.batch_job import BatchJob
+
+batch = [
+    BatchJob(task=my_task, parameters=[1, "a"]),
+    BatchJob(task=my_task, parameters=[2], parameters_keyed={"name": "b"}),
+    BatchJob(task=my_task, parameters_keyed={"id": 3, "name": "c"})
+]
+job_rids = queuer.add_jobs(batch)
+```
+
+---
+
 ## add_next_interval_func
 
 The `add_next_interval_func` method registers a custom function that determines the next execution time for scheduled jobs. This is useful for implementing complex scheduling logic beyond simple fixed intervals.
@@ -253,19 +326,20 @@ class Schedule:
 ### Job Management
 
 ```python
-# Add a single job
-job = queuer.add_job(my_task, param1, param2)
+# Add a job with both positional and keyword arguments
+job = queuer.add_job(my_task, param1, param2, paramKeyed1="test", paramKeyed2=1)
 
 # Add a job with custom options
 from model.options import Options, OnError
 options = Options(on_error=OnError(max_retries=5, timeout=60.0))
-job = queuer.add_job_with_options(options, my_task, param1)
+job = queuer.add_job_with_options(options, my_task, param1, paramKeyed1="test")
 
 # Add multiple jobs as a batch
 from model.batch_job import BatchJob
 batch = [
     BatchJob(task=my_task, parameters=[1, "a"]),
-    BatchJob(task=my_task, parameters=[2, "b"])
+    BatchJob(task=my_task, parameters=[2], parameters_keyed={"name": "b"}),
+    BatchJob(task=my_task, parameters_keyed={"id": 3, "name": "c"})
 ]
 queuer.add_jobs(batch)
 
@@ -292,6 +366,7 @@ worker_jobs = queuer.get_jobs_by_worker_rid(worker.rid)
 
 # ⭐ Features
 
+- **Keyword Arguments Support**: Python functions can use both positional arguments (`*args`) and keyword arguments (`**kwargs`), while maintaining compatibility with Go jobs that use only positional parameters.
 - **Async/Await Support**: Full asyncio integration with threading fallbacks.
 - **PostgreSQL NOTIFY/LISTEN**: Real-time job notifications without polling overhead.
 - **Batch Job Processing**: Insert job batches efficiently using PostgreSQL's `COPY FROM` feature.

{queuerpy-0.16.0 → queuerpy-0.18.0}/_version.py

@@ -1,6 +1,6 @@
 """Version information for queuerPy package."""
 
-__version__ = "0.16.0"
+__version__ = "0.18.0"
 __version_info__ = tuple(int(part) for part in __version__.split("."))
 
 # For compatibility

{queuerpy-0.16.0 → queuerpy-0.18.0}/model/worker.py

@@ -17,6 +17,7 @@ class WorkerStatus:
     READY = "READY"
     RUNNING = "RUNNING"
     FAILED = "FAILED"
+    STOPPING = "STOPPING"
     STOPPED = "STOPPED"
 
 

{queuerpy-0.16.0 → queuerpy-0.18.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "queuerPy"
-version = "0.16.0"
+version = "0.18.0"
 description = "A Python implementation of the queuer system - a job queuing and processing system with PostgreSQL backend"
 readme = "README.md"
 license = "Apache-2.0"
@@ -92,7 +92,7 @@ pythonVersion = "3.8"
 pythonPlatform = "All"
 
 [tool.mypy]
-python_version = "0.16.0"
+python_version = "0.18.0"
 warn_return_any = true
 warn_unused_configs = true
 disallow_untyped_defs = true

{queuerpy-0.16.0 → queuerpy-0.18.0}/queuer.py

@@ -28,6 +28,7 @@ from .queuer_task import QueuerTaskMixin
 from .queuer_next_interval import QueuerNextIntervalMixin
 from .queuer_listener import QueuerListenerMixin
 from .queuer_master import QueuerMasterMixin
+from .queuer_worker import QueuerWorkerMixin
 
 logger = get_logger(__name__)
 
@@ -68,6 +69,7 @@ class Queuer(
     QueuerNextIntervalMixin,
     QueuerListenerMixin,
     QueuerMasterMixin,
+    QueuerWorkerMixin,
 ):
     """
     Main queuing system class.
@@ -113,6 +115,7 @@ class Queuer(
 
         # Configuration
         self.job_poll_interval: timedelta = timedelta(minutes=5)
+        self.worker_poll_interval: timedelta = timedelta(seconds=30)
         self.retention_archive: timedelta = timedelta(days=30)
 
         # Active runners (jobs currently executing)
@@ -219,177 +222,96 @@ class Queuer(
         logger.info("Queuer started")
 
     def stop(self) -> None:
-        """Stop the queuer."""
-        # Store whether we were running to decide if we need to close connections
-        was_running = getattr(self, "_running", False)
+        """
+        Stop stops the queuer by closing the job listeners, cancelling all queued and running jobs,
+        and cancelling the context to stop the queuer.
+        """
+        # Check if already stopped (database is None means we've already cleaned up)
+        if self.database is None:
+            return  # Already stopped
+
+        # Mark as not running immediately to prevent re-entrant calls
+        was_running = self.running
+        self.running = False
 
-        # Stop tickers first to prevent new work from being queued
-        if self.heartbeat_ticker:
+        # Stop tickers first to prevent them from calling stop() recursively
+        if hasattr(self, "heartbeat_ticker") and self.heartbeat_ticker:
             try:
                 self.heartbeat_ticker.stop()
             except Exception as e:
                 logger.warning(f"Error stopping heartbeat ticker: {e}")
 
-        if self.poll_job_ticker:
+        if hasattr(self, "poll_job_ticker") and self.poll_job_ticker:
             try:
                 self.poll_job_ticker.stop()
             except Exception as e:
                 logger.warning(f"Error stopping poll job ticker: {e}")
 
-        # Close database listeners
+        # Close db listeners
         if self.job_db_listener:
             try:
-                # Use go_func but wait for completion
                 task = go_func(self.job_db_listener.stop, use_mp=False)
-                task.get_results(timeout=3.0)  # Wait up to 3 seconds total
+                task.get_results(timeout=3.0)
             except Exception as e:
                 # Only log if it's not already closed
-                if "closed" not in str(e).lower():
-                    logger.warning(f"Error stopping job listener: {e}")
-
-        # Wait for database listener runners to complete
-        if self.job_db_listener_runner:
-            try:
-                self.job_db_listener_runner.get_results(timeout=2.0)
-                self.job_db_listener_runner = None
-            except Exception as e:
-                logger.warning(f"Error waiting for job listener runner: {e}")
+                if "Listener has been closed" not in str(e):
+                    logger.error(f"Error closing job insert listener: {e}")
 
         if self.job_archive_db_listener:
             try:
                 task = go_func(self.job_archive_db_listener.stop, use_mp=False)
-                task.get_results(timeout=3.0)  # Wait up to 3 seconds total
+                task.get_results(timeout=3.0)
             except Exception as e:
-                if "closed" not in str(e).lower():
-                    logger.warning(f"Error stopping job archive listener: {e}")
+                if "Listener has been closed" not in str(e):
+                    logger.error(f"Error closing job archive listener: {e}")
 
-        # Wait for database archive listener runner to complete
-        if self.job_archive_db_listener_runner:
-            try:
-                self.job_archive_db_listener_runner.get_results(timeout=2.0)
-                self.job_archive_db_listener_runner = None
-            except Exception as e:
-                logger.warning(f"Error waiting for job archive listener runner: {e}")
-
-        # Update worker status to stopped (only if we were running)
+        # Update worker status to stopped
+        err = None
         worker_rid = None
-        if was_running:
-            try:
-                with self.worker_mutex:
+        if self.database and self.database.instance:
+            with self.worker_mutex:
+                if self.worker:
                     self.worker.status = WorkerStatus.STOPPED
                     worker_updated = self.db_worker.update_worker(self.worker)
                     if worker_updated:
+                        self.worker = worker_updated
                         worker_rid = self.worker.rid
-            except Exception as e:
-                logger.error(f"Error updating worker status to stopped: {e}")
-        else:
-            # If we were never running, just get the worker_rid for job cancellation
-            worker_rid = self.worker.rid
+                    else:
+                        err = Exception("Failed to update worker")
 
-        # Cancel all queued and running jobs
+        if err:
+            logger.error(f"Error updating worker status to stopped: {err}")
+            return
+
+        # Cancel all queued and running jobs (only if we have a valid worker RID)
         if worker_rid:
             try:
                 self.cancel_all_jobs_by_worker(worker_rid, 100)
             except Exception as e:
                 logger.error(f"Error cancelling all jobs by worker: {e}")
+                return
 
-        # Cancel the context equivalent - set cancellation event
-        if self._cancel_event:
-            try:
-                self._cancel_event.set()
-            except Exception as e:
-                logger.warning(f"Error setting cancel event: {e}")
-
-        # Signal that we're stopping
-        self.running = False
-        self._stopped.set()
-
-        # Cancel all active runners - similar to Go's job cancellation
-        active_runner_count = len(self.active_runners)
-        if active_runner_count > 0:
-            logger.info(f"Cancelling {active_runner_count} active runners")
-
+        # Cancel all active runners
         for runner_id, runner in list(self.active_runners.items()):
-            try:
-                runner.cancel()
-                if runner_id in self.active_runners:
-                    del self.active_runners[runner_id]
-                    logger.debug(f"Removed runner {runner_id} from active_runners")
+            runner.cancel()
+            del self.active_runners[runner_id]
 
-            except Exception as e:
-                logger.warning(f"Error cancelling runner {runner_id}: {e}")
-
-        # Final check
-        remaining_runners = len(self.active_runners)
-        if remaining_runners > 0:
-            logger.warning(f"{remaining_runners} runners could not be cleaned up")
-        else:
-            logger.debug("All active runners cleaned up successfully")
-
-        # Cleanup broadcasters and listeners
-        try:
-            broadcaster_count = 0
-            if self.job_insert_broadcaster:
-                listener_count = len(self.job_insert_broadcaster.listeners)
-                if listener_count > 0:
-                    logger.debug(
-                        f"Clearing {listener_count} listeners from job_insert_broadcaster"
-                    )
-                    self.job_insert_broadcaster.listeners.clear()
-                broadcaster_count += 1
-
-            if self.job_update_broadcaster:
-                listener_count = len(self.job_update_broadcaster.listeners)
-                if listener_count > 0:
-                    logger.debug(
-                        f"Clearing {listener_count} listeners from job_update_broadcaster"
-                    )
-                    self.job_update_broadcaster.listeners.clear()
-                broadcaster_count += 1
-
-            if self.job_delete_broadcaster:
-                listener_count = len(self.job_delete_broadcaster.listeners)
-                if listener_count > 0:
-                    logger.debug(
-                        f"Clearing {listener_count} listeners from job_delete_broadcaster"
-                    )
-                    self.job_delete_broadcaster.listeners.clear()
-                broadcaster_count += 1
-
-            if broadcaster_count > 0:
-                logger.debug(f"Cleaned up {broadcaster_count} broadcasters")
-
-        except Exception as e:
-            logger.warning(f"Error during broadcaster cleanup: {e}")
-
-        # Cleanup AsyncIO resources
-        try:
-            try:
-                loop = asyncio.get_running_loop()
-                pending_tasks = [
-                    task for task in asyncio.all_tasks(loop) if not task.done()
-                ]
-                if pending_tasks:
-                    logger.debug(f"Cancelling {len(pending_tasks)} pending async tasks")
-                    for task in pending_tasks:
-                        task.cancel()
-            except RuntimeError:
-                pass
-
-            # Clear the cancel event
-            self._cancel_event = None
+        # Cancel the context to stop the queuer
+        if self._cancel_event:
+            self._cancel_event.set()
 
-        except Exception as e:
-            logger.warning(f"Error during AsyncIO cleanup: {e}")
+        # Wait a moment for background goroutines to finish gracefully
+        time.sleep(0.1)
 
+        # Close database connection
         if self.database:
+            logger.info("Closing database connection")
             try:
-                logger.info(f"Closing database '{self.name}'")
                 self.database.close()
             except Exception as e:
-                logger.error(f"Error closing main database connection: {e}")
+                logger.error(f"Error closing database connection: {e}")
 
-        logger.info(f"Queuer '{self.worker.name}' stopped")
+        logger.info("Queuer stopped")
 
     # Job notification listeners
     async def _handle_job_notification(self, notification: str) -> None:
@@ -524,31 +446,77 @@ class Queuer(
 
     # Tickers
     def _heartbeat_func(self) -> None:
-        """Send periodic heartbeats - only updates database, not queuer state."""
+        """Send periodic heartbeats and handle worker status changes."""
         try:
-            # Get current worker with mutex
+            logger.debug("Sending worker heartbeat...")
+
+            # Get current worker with read lock
             with self.worker_mutex:
-                current_worker = self.worker
-
-            updated_worker: Optional[Worker] = None
-            if current_worker:
-                # Update timestamp and save to database
-                current_worker.updated_at = datetime.now()
-                updated_worker = self.db_worker.update_worker(current_worker)
-                logger.debug(
-                    f"Updated worker heartbeat timestamp: {current_worker.updated_at}"
+                worker = self.worker
+
+            if worker is None:
+                return
+
+            # Select worker from database for heartbeat
+            worker_from_db = self.db_worker.select_worker(worker.rid)
+            if not worker_from_db:
+                logger.error("Error selecting worker for heartbeat")
+                return
+
+            # Handle worker status
+            if worker_from_db.status == WorkerStatus.STOPPED:
+                logger.info(
+                    f"Stopping worker... (worker_status: {worker_from_db.status})"
                 )
+                try:
+                    self.stop()
+                except Exception as e:
+                    logger.error(f"Error stopping queuer: {e}")
+                return
+
+            elif worker_from_db.status == WorkerStatus.STOPPING:
+                if worker_from_db.max_concurrency != 0:
+                    logger.info(
+                        f"Gracefully stopping worker... (worker_status: {worker_from_db.status})"
+                    )
+                    worker_from_db.max_concurrency = 0
+                    worker_from_db = self.db_worker.update_worker(worker_from_db)
+                    if not worker_from_db:
+                        logger.error("Error updating worker concurrency")
+                        return
+                elif len(self.active_runners) == 0:
+                    logger.info(
+                        f"All running jobs finished, stopping worker... (worker_status: {worker_from_db.status})"
+                    )
+                    worker_from_db.status = WorkerStatus.STOPPED
+                    worker_from_db = self.db_worker.update_worker(worker_from_db)
+                    if not worker_from_db:
+                        logger.error("Error updating worker status to stopped")
+                        return
+                    try:
+                        self.stop()
+                    except Exception as e:
+                        logger.error(f"Error stopping queuer: {e}")
+                    return
+
+            else:
+                # Default case: update worker heartbeat
+                worker_from_db = self.db_worker.update_worker(worker)
+                if not worker_from_db:
+                    logger.error("Error updating worker heartbeat")
+                    return
+
+            # Update local worker with write lock
+            with self.worker_mutex:
+                self.worker = worker_from_db
 
-            if updated_worker:
-                with self.worker_mutex:
-                    self.worker = updated_worker
         except Exception as e:
             logger.error(f"Heartbeat error: {e}")
 
     def _start_heartbeat_ticker(self) -> None:
         """Start heartbeat ticker using threading."""
         self.heartbeat_ticker = Ticker(
-            timedelta(seconds=30),
+            self.worker_poll_interval,
             self._heartbeat_func,
             use_mp=False,
         )

{queuerpy-0.16.0 → queuerpy-0.18.0/queuerPy.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: queuerPy
-Version: 0.16.0
+Version: 0.18.0
 Summary: A Python implementation of the queuer system - a job queuing and processing system with PostgreSQL backend
 Author-email: Simon Herrmann <siherrmann@users.noreply.github.com>
 Maintainer-email: Simon Herrmann <siherrmann@users.noreply.github.com>
@@ -43,9 +43,10 @@ Dynamic: license-file
 
 # queuerPy
 
+[![PyPi](https://pythonico.leapcell.app/pypi/queuerPy.svg?style=shields&data=n,v,d)](https://pypi.org/project/queuerPy/)
 [![Python](https://img.shields.io/badge/Python-3.10+-blue.svg)](https://www.python.org/)
-[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/siherrmann/queuer/blob/master/LICENSE)
 ![Coverage](https://raw.githubusercontent.com/siherrmann/queuerPy/refs/heads/main/coverage-badge.svg)
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/siherrmann/queuer/blob/master/LICENSE)
 
 Python port of the queuer package - a queueing system based on PostgreSQL.
 
@@ -97,9 +98,15 @@ def example_task():
 That's easy, right? Adding a job is just as easy:
 
 ```python
-# Add a job to the queue
+# Add a job to the queue with positional arguments
 job = q.add_job(example_task, 5, "12")
 print(f"Job added: {job.rid}")
+
+# Add a job with keyword arguments (kwargs)
+job = q.add_job(example_task, paramKeyed1="test", debug=True)
+
+# Add a job with both positional and keyword arguments
+job = q.add_job(example_task, 5, "12", paramKeyed1="test", debug=True)
 ```
 
 In the initialisation of the queuer the existence of the necessary database tables is checked and if they don't exist they get created. The database is configured with these environment variables:
@@ -199,6 +206,72 @@ This method handles the registration of a task, making the worker able to pick u
 
 ---
 
+## add_job
+
+The `add_job` method adds a new job to the queue for execution. Jobs are units of work that will be processed by the queuer.
+
+```python
+def add_job(
+    self, 
+    task: Union[Callable, str], 
+    *parameters: Any,
+    **parameters_keyed: Any
+) -> Job
+
+def add_job_with_options(
+    self,
+    options: Optional[Options],
+    task: Union[Callable, str],
+    *parameters: Any,
+    **parameters_keyed: Any
+) -> Job
+```
+
+- `task`: A `Callable` or `str` representing the task to execute. If a callable, it must be registered with `add_task` first.
+- `options`: Optional `Options` for custom error handling or scheduling behavior. Only available in `add_job_with_options()`.
+- `*parameters`: Positional arguments to pass to the task function.
+- `**parameters_keyed`: Keyword arguments to pass to the task function (Python-specific feature).
+
+**Examples:**
+
+```python
+# Job with both positional and keyword arguments
+job = queuer.add_job(my_task, "arg1", paramKeyed1="test", paramKeyed2=1)
+
+# Job with custom options (requires add_job_with_options)
+options = Options(on_error=OnError(max_retries=5))
+job = queuer.add_job_with_options(options, my_task, "arg1", debug=True)
+```
+
+**Note:** Keyword arguments (`**parameters_keyed`) are stored separately in the database and enable Python functions to be called with named parameters. This is useful for optional parameters, default values, and improved code clarity. Go jobs continue to use only positional parameters.
+
+---
+
+## add_jobs
+
+The `add_jobs` method allows efficient batch insertion of multiple jobs at once.
+
+```python
+def add_jobs(self, batch_jobs: List[BatchJob]) -> List[UUID]
+```
+
+- `batch_jobs`: A list of `BatchJob` instances to insert.
+
+**Example:**
+
+```python
+from model.batch_job import BatchJob
+
+batch = [
+    BatchJob(task=my_task, parameters=[1, "a"]),
+    BatchJob(task=my_task, parameters=[2], parameters_keyed={"name": "b"}),
+    BatchJob(task=my_task, parameters_keyed={"id": 3, "name": "c"})
+]
+job_rids = queuer.add_jobs(batch)
+```
+
+---
+
 ## add_next_interval_func
 
 The `add_next_interval_func` method registers a custom function that determines the next execution time for scheduled jobs. This is useful for implementing complex scheduling logic beyond simple fixed intervals.
@@ -296,19 +369,20 @@ class Schedule:
 ### Job Management
 
 ```python
-# Add a single job
-job = queuer.add_job(my_task, param1, param2)
+# Add a job with both positional and keyword arguments
+job = queuer.add_job(my_task, param1, param2, paramKeyed1="test", paramKeyed2=1)
 
 # Add a job with custom options
 from model.options import Options, OnError
 options = Options(on_error=OnError(max_retries=5, timeout=60.0))
-job = queuer.add_job_with_options(options, my_task, param1)
+job = queuer.add_job_with_options(options, my_task, param1, paramKeyed1="test")
 
 # Add multiple jobs as a batch
 from model.batch_job import BatchJob
 batch = [
     BatchJob(task=my_task, parameters=[1, "a"]),
-    BatchJob(task=my_task, parameters=[2, "b"])
+    BatchJob(task=my_task, parameters=[2], parameters_keyed={"name": "b"}),
+    BatchJob(task=my_task, parameters_keyed={"id": 3, "name": "c"})
 ]
 queuer.add_jobs(batch)
 
@@ -335,6 +409,7 @@ worker_jobs = queuer.get_jobs_by_worker_rid(worker.rid)
 
 # ⭐ Features
 
+- **Keyword Arguments Support**: Python functions can use both positional arguments (`*args`) and keyword arguments (`**kwargs`), while maintaining compatibility with Go jobs that use only positional parameters.
 - **Async/Await Support**: Full asyncio integration with threading fallbacks.
 - **PostgreSQL NOTIFY/LISTEN**: Real-time job notifications without polling overhead.
 - **Batch Job Processing**: Insert job batches efficiently using PostgreSQL's `COPY FROM` feature.

{queuerpy-0.16.0 → queuerpy-0.18.0}/queuerPy.egg-info/SOURCES.txt

@@ -12,6 +12,7 @@ queuer_listener.py
 queuer_master.py
 queuer_next_interval.py
 queuer_task.py
+queuer_worker.py
 ./__init__.py
 ./_version.py
 ./py.typed
@@ -22,6 +23,7 @@ queuer_task.py
 ./queuer_master.py
 ./queuer_next_interval.py
 ./queuer_task.py
+./queuer_worker.py
 ./core/__init__.py
 ./core/broadcaster.py
 ./core/listener.py

queuerpy-0.18.0/queuer_worker.py

@@ -0,0 +1,96 @@
+"""
+Worker-related methods for the Python queuer implementation.
+Mirrors Go's queuerWorker.go functionality.
+"""
+
+import logging
+from typing import Optional
+from uuid import UUID
+
+from .helper.error import QueuerError
+from .model.worker import Worker, WorkerStatus
+from .queuer_global import QueuerGlobalMixin
+
+# Set up logger
+logger = logging.getLogger(__name__)
+
+
+class QueuerWorkerMixin(QueuerGlobalMixin):
+    """
+    Mixin class containing worker-related methods for the Queuer.
+    This mirrors the worker methods from Go's queuerWorker.go.
+    """
+
+    def __init__(self):
+        super().__init__()
+
+    def stop_worker(self, worker_rid: UUID) -> None:
+        """
+        StopWorkerGracefully sets the status of the specified worker to 'STOPPED'
+        to cancel running jobs when stopping.
+
+        :param worker_rid: The RID of the worker to stop
+        :raises QueuerError: If getting or updating the worker fails
+        """
+        try:
+            worker = self.get_worker(worker_rid)
+            if worker is None:
+                raise ValueError(f"Worker {worker_rid} not found")
+
+            worker.status = WorkerStatus.STOPPED
+
+            worker_updated = self.db_worker.update_worker(worker)
+            if not worker_updated:
+                raise ValueError(
+                    f"Failed to update worker {worker_rid} status to stopped"
+                )
+
+            # Update local worker object if this is the current queuer's worker
+            if self.worker is not None and self.worker.rid == worker_rid:
+                with self.worker_mutex:
+                    self.worker = worker_updated
+
+        except Exception as e:
+            raise QueuerError("stopping worker", e)
+
+    def stop_worker_gracefully(self, worker_rid: UUID) -> None:
+        """
+        StopWorkerGracefully sets the worker's status to STOPPING
+        to allow it to finish current tasks before stopping.
+
+        :param worker_rid: The RID of the worker to stop gracefully
+        :raises QueuerError: If getting or updating the worker fails
+        """
+        try:
+            worker = self.get_worker(worker_rid)
+            if worker is None:
+                raise ValueError(f"Worker {worker_rid} not found")
+
+            worker.status = WorkerStatus.STOPPING
+
+            worker_updated = self.db_worker.update_worker(worker)
+            if not worker_updated:
+                raise ValueError(
+                    f"Failed to update worker {worker_rid} status to stopping"
+                )
+
+            # Update local worker object if this is the current queuer's worker
+            if self.worker is not None and self.worker.rid == worker_rid:
+                with self.worker_mutex:
+                    self.worker = worker_updated
+
+        except Exception as e:
+            raise QueuerError("stopping worker gracefully", e)
+
+    def get_worker(self, worker_rid: UUID) -> Optional[Worker]:
+        """
+        Get a worker by its RID.
+
+        :param worker_rid: The RID of the worker to get
+        :returns: The worker if found, None otherwise
+        :raises QueuerError: If the database query fails
+        """
+        try:
+            return self.db_worker.select_worker(worker_rid)
+        except Exception as e:
+            raise QueuerError("getting worker", e)