queuemgr 1.0.0__tar.gz

queuemgr-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Vasiliy Zdanovskiy
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

queuemgr-1.0.0/MANIFEST.in

@@ -0,0 +1,15 @@
+include README.md
+include LICENSE
+include SERVICE_README.md
+include QUICK_START.md
+include requirements.txt
+include install.sh
+include queuemgr.service
+recursive-include queuemgr/examples *.py
+recursive-include docs *.md
+recursive-include tests *.py
+global-exclude __pycache__
+global-exclude *.py[co]
+global-exclude *.log
+global-exclude *.pid
+global-exclude *.jsonl

queuemgr-1.0.0/PKG-INFO

@@ -0,0 +1,148 @@
+Metadata-Version: 2.4
+Name: queuemgr
+Version: 1.0.0
+Summary: Full-featured job queue system with multiprocessing support for Linux
+Home-page: https://github.com/vasilyvz/queuemgr
+Author: Vasiliy Zdanovskiy
+Author-email: Vasiliy Zdanovskiy <vasilyvz@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/vasilyvz/queuemgr
+Project-URL: Repository, https://github.com/vasilyvz/queuemgr
+Project-URL: Documentation, https://github.com/vasilyvz/queuemgr#readme
+Project-URL: Bug Tracker, https://github.com/vasilyvz/queuemgr/issues
+Keywords: job-queue,multiprocessing,task-scheduler,linux,systemd
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: System Administrators
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: System :: Distributed Computing
+Classifier: Topic :: System :: Systems Administration
+Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
+Classifier: Topic :: Scientific/Engineering
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+Requires-Dist: black>=23.0.0; extra == "dev"
+Requires-Dist: flake8>=6.0.0; extra == "dev"
+Requires-Dist: mypy>=1.0.0; extra == "dev"
+Provides-Extra: web
+Requires-Dist: flask>=2.0.0; extra == "web"
+Provides-Extra: examples
+Requires-Dist: requests>=2.25.0; extra == "examples"
+Dynamic: author
+Dynamic: home-page
+Dynamic: license-file
+Dynamic: requires-python
+
+# Queue Manager
+
+A Python-based job queue system with per-job processes and signal variables.
+
+## Features
+
+- **Per-Job Processes**: Each job runs in its own OS process for isolation
+- **Signal Variables**: Jobs communicate via shared state protected by mutexes
+- **Registry Persistence**: Job states and results are persisted to a registry
+- **Process Control**: Start, stop, and delete jobs with graceful termination
+- **Progress Tracking**: Real-time progress updates and status monitoring
+- **Error Handling**: Comprehensive error handling and recovery mechanisms
+
+## Installation
+
+```bash
+pip install queuemgr
+```
+
+## Quick Start
+
+```python
+from queuemgr.queue.job_queue import JobQueue
+from queuemgr.core.registry import JsonlRegistry
+from queuemgr.jobs.base import QueueJobBase
+from queuemgr.core.types import JobStatus
+from queuemgr.core.ipc import update_job_state
+
+class MyJob(QueueJobBase):
+    def execute(self):
+        # Your job logic here
+        for i in range(100):
+            update_job_state(
+                self._shared_state,
+                progress=i,
+                description=f"Processing item {i}"
+            )
+    
+    def on_start(self):
+        update_job_state(self._shared_state, description="Job started")
+    
+    def on_stop(self):
+        update_job_state(self._shared_state, description="Job stopped")
+    
+    def on_end(self):
+        update_job_state(self._shared_state, description="Job completed")
+    
+    def on_error(self, exc):
+        update_job_state(self._shared_state, description=f"Job failed: {exc}")
+
+# Create queue and job
+registry = JsonlRegistry("my_registry.jsonl")
+queue = JobQueue(registry)
+job = MyJob("my-job-1", {"param": "value"})
+
+# Add and start job
+queue.add_job(job)
+queue.start_job("my-job-1")
+
+# Monitor progress
+status = queue.get_job_status("my-job-1")
+print(f"Status: {status.status}, Progress: {status.progress}%")
+```
+
+## Examples
+
+See the `examples/` directory for complete examples:
+
+- `simple_job.py` - Basic job creation and execution
+- `error_handling_job.py` - Error handling and recovery
+- `progress_job.py` - Long-running job with progress updates
+- `registry_example.py` - Registry functionality and job history
+
+## Development
+
+```bash
+# Install development dependencies
+pip install -e ".[dev]"
+
+# Run tests
+pytest
+
+# Run tests with coverage
+pytest --cov=queuemgr
+
+# Format code
+black queuemgr tests
+
+# Lint code
+flake8 queuemgr tests
+
+# Type check
+mypy queuemgr
+```
+
+## License
+
+MIT License - see LICENSE file for details.
+
+## Author
+
+Vasiliy Zdanovskiy - vasilyvz@gmail.com
+# vvz-queuemgr

queuemgr-1.0.0/QUICK_START.md

@@ -0,0 +1,118 @@
+# 🚀 Queue Manager - Быстрый старт
+
+## Что это?
+
+Queue Manager - это полнофункциональная система управления очередями заданий для Linux с поддержкой multiprocessing, systemd интеграцией и веб-интерфейсом.
+
+## ⚡ Быстрая установка
+
+```bash
+# Клонируйте репозиторий
+git clone https://github.com/YOUR_USERNAME/queuemgr.git
+cd queuemgr
+
+# Установите сервис (требуются права root)
+sudo ./install.sh
+```
+
+## 🎯 Основные возможности
+
+- **🚀 Автоматическое управление процессами** - каждое задание выполняется в отдельном процессе
+- **📊 Мониторинг в реальном времени** - отслеживание статуса и прогресса заданий
+- **🖥️ Множественные интерфейсы** - CLI, веб-интерфейс, systemd сервис
+- **⚙️ Systemd интеграция** - автозапуск и управление через systemctl
+- **🔧 Обработка ошибок** - автоматическое восстановление и graceful shutdown
+- **📈 Масштабируемость** - поддержка множества параллельных заданий
+
+## 🛠️ Типы заданий
+
+- **Обработка данных** - большие датасеты, ETL процессы
+- **Файловые операции** - копирование, перемещение, архивирование
+- **API вызовы** - интеграция с внешними сервисами
+- **Операции с БД** - запросы, обновления, миграции
+- **Мониторинг** - сбор метрик, алерты, отчеты
+
+## 📋 Управление сервисом
+
+```bash
+# Статус сервиса
+sudo systemctl status queuemgr
+
+# Запуск/остановка
+sudo systemctl start queuemgr
+sudo systemctl stop queuemgr
+
+# Просмотр логов
+sudo journalctl -u queuemgr -f
+```
+
+## 🖥️ Интерфейсы
+
+### CLI
+```bash
+# Управление заданиями
+/opt/queuemgr/.venv/bin/python -m queuemgr.service.cli job list
+/opt/queuemgr/.venv/bin/python -m queuemgr.service.cli job add
+/opt/queuemgr/.venv/bin/python -m queuemgr.service.cli monitor
+```
+
+### Веб-интерфейс
+```bash
+# Запуск веб-интерфейса
+/opt/queuemgr/.venv/bin/python -m queuemgr.service.web
+
+# Откройте в браузере: http://localhost:5000
+```
+
+## 📚 Примеры использования
+
+```python
+from queuemgr.proc_api import proc_queue_system
+from queuemgr.jobs.base import QueueJobBase
+
+class MyJob(QueueJobBase):
+    def execute(self):
+        print(f"Выполняю задание {self.job_id}")
+        # Ваша логика здесь
+        
+    def on_start(self):
+        print("Задание запущено")
+        
+    def on_end(self):
+        print("Задание завершено")
+
+# Использование
+with proc_queue_system() as queue:
+    queue.add_job(MyJob, "my-job-1", {"param": "value"})
+    queue.start_job("my-job-1")
+```
+
+## 📖 Документация
+
+- [Полная документация](SERVICE_README.md)
+- [Техническая спецификация](docs/tech_spec.md)
+- [Примеры использования](queuemgr/examples/)
+
+## 🎯 Готово к продакшну
+
+- ✅ Все тесты проходят
+- ✅ Код отформатирован и проверен
+- ✅ Полная документация
+- ✅ Примеры использования
+- ✅ Systemd интеграция
+- ✅ Обработка ошибок
+
+## 🤝 Вклад в проект
+
+1. Fork репозитория
+2. Создайте feature branch
+3. Внесите изменения
+4. Создайте Pull Request
+
+## 📄 Лицензия
+
+MIT License - см. файл LICENSE
+
+---
+
+**Queue Manager** - надежная система управления очередями заданий для Linux! 🚀

queuemgr-1.0.0/README.md

@@ -0,0 +1,103 @@
+# Queue Manager
+
+A Python-based job queue system with per-job processes and signal variables.
+
+## Features
+
+- **Per-Job Processes**: Each job runs in its own OS process for isolation
+- **Signal Variables**: Jobs communicate via shared state protected by mutexes
+- **Registry Persistence**: Job states and results are persisted to a registry
+- **Process Control**: Start, stop, and delete jobs with graceful termination
+- **Progress Tracking**: Real-time progress updates and status monitoring
+- **Error Handling**: Comprehensive error handling and recovery mechanisms
+
+## Installation
+
+```bash
+pip install queuemgr
+```
+
+## Quick Start
+
+```python
+from queuemgr.queue.job_queue import JobQueue
+from queuemgr.core.registry import JsonlRegistry
+from queuemgr.jobs.base import QueueJobBase
+from queuemgr.core.types import JobStatus
+from queuemgr.core.ipc import update_job_state
+
+class MyJob(QueueJobBase):
+    def execute(self):
+        # Your job logic here
+        for i in range(100):
+            update_job_state(
+                self._shared_state,
+                progress=i,
+                description=f"Processing item {i}"
+            )
+    
+    def on_start(self):
+        update_job_state(self._shared_state, description="Job started")
+    
+    def on_stop(self):
+        update_job_state(self._shared_state, description="Job stopped")
+    
+    def on_end(self):
+        update_job_state(self._shared_state, description="Job completed")
+    
+    def on_error(self, exc):
+        update_job_state(self._shared_state, description=f"Job failed: {exc}")
+
+# Create queue and job
+registry = JsonlRegistry("my_registry.jsonl")
+queue = JobQueue(registry)
+job = MyJob("my-job-1", {"param": "value"})
+
+# Add and start job
+queue.add_job(job)
+queue.start_job("my-job-1")
+
+# Monitor progress
+status = queue.get_job_status("my-job-1")
+print(f"Status: {status.status}, Progress: {status.progress}%")
+```
+
+## Examples
+
+See the `examples/` directory for complete examples:
+
+- `simple_job.py` - Basic job creation and execution
+- `error_handling_job.py` - Error handling and recovery
+- `progress_job.py` - Long-running job with progress updates
+- `registry_example.py` - Registry functionality and job history
+
+## Development
+
+```bash
+# Install development dependencies
+pip install -e ".[dev]"
+
+# Run tests
+pytest
+
+# Run tests with coverage
+pytest --cov=queuemgr
+
+# Format code
+black queuemgr tests
+
+# Lint code
+flake8 queuemgr tests
+
+# Type check
+mypy queuemgr
+```
+
+## License
+
+MIT License - see LICENSE file for details.
+
+## Author
+
+Vasiliy Zdanovskiy - vasilyvz@gmail.com
+# vvz-queuemgr