processpype 0.1.0__tar.gz

processpype-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Gianluca Pagliara
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

processpype-0.1.0/PKG-INFO

@@ -0,0 +1,24 @@
+Metadata-Version: 2.1
+Name: processpype
+Version: 0.1.0
+Summary: A modular application framework with built-in FastAPI integration and pluggable services
+Author: Gianluca Pagliara
+Author-email: pagliara.gianluca@gmail.com
+Requires-Python: >=3.13,<4.0
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.13
+Requires-Dist: fastapi (>=0.115.6,<0.116.0)
+Requires-Dist: httpx (>=0.28.1,<0.29.0)
+Requires-Dist: logfire[fastapi] (>=2.11.1,<3.0.0)
+Requires-Dist: psutil (>=6.1.1,<7.0.0)
+Requires-Dist: pydantic (>=2.10.4,<3.0.0)
+Requires-Dist: pytz (>=2024.2,<2025.0)
+Requires-Dist: pyyaml (>=6.0.2,<7.0.0)
+Requires-Dist: telethon (>=1.38.1,<2.0.0)
+Requires-Dist: uvicorn (>=0.34.0,<0.35.0)
+Description-Content-Type: text/markdown
+
+# Processpype
+
+Processpype is a framework for building and managing an application that is composed of services.
+

processpype-0.1.0/README.md

@@ -0,0 +1,3 @@
+# Processpype
+
+Processpype is a framework for building and managing an application that is composed of services.

processpype-0.1.0/processpype/core/README.md

@@ -0,0 +1,232 @@
+# ProcessPype Core Module
+
+The core module provides the fundamental building blocks for creating and managing services in the ProcessPype framework. This document outlines the architecture, components, and guides for implementing new services.
+
+## Architecture Overview
+
+### Core Components
+
+#### 1. Application (`application.py`)
+The central orchestrator that manages the lifecycle of the application and its services.
+
+```python
+from processpype.core import Application
+
+app = await Application.create("config.yaml")
+await app.start()
+```
+
+Key features:
+- Async context manager support
+- FastAPI integration
+- Service lifecycle management
+- Configuration management
+
+#### 2. Application Manager (`manager.py`)
+Handles service registration, state management, and lifecycle operations.
+
+Key responsibilities:
+- Service registration and retrieval
+- Service state management
+- Service startup/shutdown orchestration
+
+#### 3. Router (`router.py`)
+Provides REST API endpoints for application and service management.
+
+Available endpoints:
+- `GET /` - Application status
+- `GET /services` - List registered services
+- `POST /services/{service_name}/start` - Start a service
+- `POST /services/{service_name}/stop` - Stop a service
+
+#### 4. Models (`models.py`)
+Core data models and enums for the application.
+
+```python
+from processpype.core.models import ServiceState
+
+# Available states
+ServiceState.STOPPED
+ServiceState.STARTING
+ServiceState.RUNNING
+ServiceState.STOPPING
+ServiceState.ERROR
+```
+
+#### 5. Configuration (`config/`)
+Handles application and service configuration management using Pydantic models.
+
+## Implementing New Services
+
+### 1. Basic Service Structure
+
+```python
+from processpype.core.service import Service
+from processpype.core.models import ServiceState
+
+class MyService(Service):
+    def __init__(self, name: str | None = None):
+        super().__init__(name or "my_service")
+
+    async def start(self) -> None:
+        self.set_state(ServiceState.STARTING)
+        # Initialize your service
+        self.set_state(ServiceState.RUNNING)
+
+    async def stop(self) -> None:
+        self.set_state(ServiceState.STOPPING)
+        # Cleanup resources
+        self.set_state(ServiceState.STOPPED)
+```
+
+### 2. Adding Configuration
+
+```python
+from pydantic import BaseModel
+from processpype.core.config.models import ServiceConfiguration
+
+class MyServiceConfig(ServiceConfiguration):
+    custom_field: str
+    port: int = 8080
+
+class MyService(Service):
+    def configure(self, config: MyServiceConfig) -> None:
+        self._config = config
+        # Apply configuration
+```
+
+### 3. Adding API Routes
+
+```python
+from fastapi import APIRouter
+
+class MyService(Service):
+    def __init__(self, name: str | None = None):
+        super().__init__(name or "my_service")
+        self._router = APIRouter(prefix=f"/services/{self.name}")
+        self._setup_routes()
+
+    def _setup_routes(self) -> None:
+        @self._router.get("/status")
+        async def get_status():
+            return {"state": self.state}
+```
+
+### 4. Error Handling
+
+```python
+from processpype.core.models import ServiceState
+
+class MyService(Service):
+    async def start(self) -> None:
+        try:
+            self.set_state(ServiceState.STARTING)
+            # Initialize
+            self.set_state(ServiceState.RUNNING)
+        except Exception as e:
+            self.set_error(str(e))
+            self.set_state(ServiceState.ERROR)
+            raise
+```
+
+## Service Lifecycle
+
+1. **Registration**
+```python
+app = await Application.create("config.yaml")
+service = app.register_service(MyService)
+```
+
+2. **Configuration**
+```yaml
+# config.yaml
+services:
+  my_service:
+    enabled: true
+    custom_field: "value"
+    port: 8080
+```
+
+3. **Startup**
+- Service state transitions: STOPPED → STARTING → RUNNING
+- Configuration is applied
+- Resources are initialized
+- API routes are registered
+
+4. **Runtime**
+- Service handles requests
+- Maintains state
+- Reports health status
+
+5. **Shutdown**
+- Service state transitions: RUNNING → STOPPING → STOPPED
+- Resources are cleaned up
+- API routes are unregistered
+
+## Best Practices
+
+1. **State Management**
+- Always use `set_state()` for state transitions
+- Handle errors appropriately with `set_error()`
+- Check state before operations
+
+2. **Configuration**
+- Use Pydantic models for configuration
+- Provide sensible defaults
+- Validate configuration in `configure()`
+
+3. **Resource Management**
+- Initialize resources in `start()`
+- Clean up resources in `stop()`
+- Use async context managers when possible
+
+4. **Error Handling**
+- Catch and handle exceptions appropriately
+- Set service state to ERROR on failures
+- Provide meaningful error messages
+
+5. **API Design**
+- Use FastAPI best practices
+- Prefix routes with service name
+- Provide OpenAPI documentation
+
+## Logging
+
+The framework uses structured logging via `logfire.py`:
+
+```python
+from processpype.core.logfire import get_service_logger
+
+class MyService(Service):
+    def __init__(self, name: str | None = None):
+        super().__init__(name or "my_service")
+        self.logger = get_service_logger(self.name)
+
+    async def start(self) -> None:
+        self.logger.info("Starting service", extra={"config": self._config.dict()})
+```
+
+## Testing Services
+
+1. **Unit Tests**
+```python
+async def test_my_service():
+    service = MyService()
+    await service.start()
+    assert service.state == ServiceState.RUNNING
+    await service.stop()
+    assert service.state == ServiceState.STOPPED
+```
+
+2. **Integration Tests**
+```python
+from fastapi.testclient import TestClient
+
+async def test_my_service_api():
+    app = await Application.create()
+    service = app.register_service(MyService)
+    client = TestClient(app.api)
+
+    response = client.get("/services/my_service/status")
+    assert response.status_code == 200
+```

processpype-0.1.0/processpype/core/__init__.py

@@ -0,0 +1,13 @@
+"""Core module for ProcessPype."""
+
+from .application import Application
+from .models import ApplicationStatus, ServiceState, ServiceStatus
+from .service.service import Service
+
+__all__ = [
+    "Application",
+    "Service",
+    "ServiceState",
+    "ServiceStatus",
+    "ApplicationStatus",
+]

processpype-0.1.0/processpype/core/application.py

@@ -0,0 +1,231 @@
+"""Core application class for ProcessPype."""
+
+import asyncio
+from types import TracebackType
+from typing import Any
+
+import uvicorn
+from fastapi import FastAPI
+
+from processpype.core.manager import ApplicationManager
+from processpype.core.system import setup_timezone
+
+from .configuration import ConfigurationManager
+from .configuration.models import ApplicationConfiguration
+from .logfire import get_service_logger, setup_logfire
+from .models import ServiceState
+from .router import ApplicationRouter
+from .service import Service
+
+
+class Application:
+    """Core application with built-in FastAPI integration."""
+
+    def __init__(self, config: ApplicationConfiguration):
+        """Initialize the application.
+
+        Args:
+            config: Application configuration
+        """
+        self._config = config
+        self._initialized = False
+        self._lock = asyncio.Lock()
+        self._manager: ApplicationManager | None = None
+
+    @classmethod
+    async def create(
+        cls, config_file: str | None = None, **kwargs: Any
+    ) -> "Application":
+        """Create application instance with configuration from file and/or kwargs.
+
+        Args:
+            config_file: Optional path to configuration file
+            **kwargs: Configuration overrides
+
+        Returns:
+            Application instance
+        """
+        config = await ConfigurationManager.load_application_config(
+            config_file=config_file, **kwargs
+        )
+        return cls(config)
+
+    # === Properties ===
+
+    @property
+    def is_initialized(self) -> bool:
+        """Check if the application is initialized."""
+        return self._initialized
+
+    @property
+    def config(self) -> ApplicationConfiguration:
+        """Get application configuration."""
+        return self._config
+
+    # === Lifecycle ===
+
+    async def start(self) -> None:
+        """Start the application and API server."""
+        if not self.is_initialized:
+            await self.initialize()
+
+        if not self._manager:
+            raise RuntimeError("Application manager not initialized")
+
+        self._manager.set_state(ServiceState.STARTING)
+        self.logger.info(
+            f"Starting application on {self._config.host}:{self._config.port}"
+        )
+
+        # Start enabled services
+        await self._manager.start_enabled_services()
+
+        # Start uvicorn server
+        config = uvicorn.Config(
+            self.api,
+            host=self._config.host,
+            port=self._config.port,
+            log_level="debug" if self._config.debug else "info",
+        )
+        server = uvicorn.Server(config)
+
+        try:
+            self._manager.set_state(ServiceState.RUNNING)
+            await server.serve()
+        except Exception as e:
+            self._manager.set_state(ServiceState.ERROR)
+            self.logger.error(f"Failed to start application: {e}")
+            raise
+
+    async def stop(self) -> None:
+        """Stop the application and all services."""
+        if not self.is_initialized or not self._manager:
+            return
+
+        self._manager.set_state(ServiceState.STOPPING)
+        self.logger.info("Stopping application")
+
+        # Stop all services
+        await self._manager.stop_all_services()
+        self._manager.set_state(ServiceState.STOPPED)
+
+    async def __aenter__(self) -> "Application":
+        """Async context manager entry."""
+        await self.initialize()
+        return self
+
+    async def __aexit__(
+        self,
+        exc_type: type[BaseException] | None,
+        exc_val: BaseException | None,
+        exc_tb: TracebackType | None,
+    ) -> None:
+        """Async context manager exit."""
+        await self.stop()
+
+    # === Initialization ===
+
+    async def initialize(self) -> None:
+        """Initialize the application asynchronously."""
+        async with self._lock:
+            if self.is_initialized:
+                return
+
+            setup_timezone()
+
+            # Initialize FastAPI
+            self.api = FastAPI(title=self._config.title, version=self._config.version)
+
+            # Setup logging
+            setup_logfire(
+                self.api,
+                token=self._config.logfire_key,
+                environment=self._config.environment,
+            )
+            self.logger = get_service_logger("application")
+
+            # Initialize manager
+            self.initialize_manager()
+
+            # Setup core routes
+            self.initialize_router()
+            self.logger.info(
+                "Application initialized",
+                extra={
+                    "host": self._config.host,
+                    "port": self._config.port,
+                    "version": self._config.version,
+                    "environment": self._config.environment,
+                },
+            )
+
+            self._initialized = True
+
+    def initialize_manager(self) -> None:
+        """Initialize the application manager."""
+        self._manager = ApplicationManager(self.logger, self._config)
+        self._manager.set_state(ServiceState.INITIALIZED)
+
+    def initialize_router(self) -> None:
+        if self._manager is None:
+            raise RuntimeError("Application manager not initialized")
+
+        router = ApplicationRouter(
+            get_version=lambda: self._config.version,
+            get_state=lambda: self._manager.state
+            if self._manager
+            else ServiceState.STOPPED,
+            get_services=lambda: self._manager.services if self._manager else {},
+            start_service=self._manager.start_service,
+            stop_service=self._manager.stop_service,
+        )
+        self.api.include_router(router)
+
+    # === Service Management ===
+
+    def register_service(
+        self, service_class: type[Service], name: str | None = None
+    ) -> Service:
+        """Register a new service.
+
+        Args:
+            service_class: Service class to register
+            name: Optional service name override
+
+        Returns:
+            The registered service instance
+
+        Raises:
+            RuntimeError: If application is not initialized
+            ValueError: If service name is already registered
+        """
+        if not self.is_initialized or not self._manager:
+            raise RuntimeError(
+                "Application must be initialized before registering services"
+            )
+
+        service = self._manager.register_service(service_class, name)
+        if service.router:
+            self.api.include_router(service.router)
+
+        return service
+
+    def get_service(self, name: str) -> Service | None:
+        """Get a service by name."""
+        if not self._manager:
+            return None
+        return self._manager.get_service(name)
+
+    async def start_service(self, service_name: str) -> None:
+        """Start a service by name."""
+        if not self.is_initialized or not self._manager:
+            raise RuntimeError(
+                "Application must be initialized before starting services"
+            )
+        await self._manager.start_service(service_name)
+
+    async def stop_service(self, service_name: str) -> None:
+        """Stop a service by name."""
+        if not self._manager:
+            return
+        await self._manager.stop_service(service_name)

processpype-0.1.0/processpype/core/configuration/__init__.py

@@ -0,0 +1,14 @@
+"""Configuration management for ProcessPype."""
+
+from .manager import ConfigurationManager
+from .models import ConfigurationModel, ServiceConfiguration
+from .providers import ConfigurationProvider, EnvProvider, FileProvider
+
+__all__ = [
+    "ConfigurationManager",
+    "ConfigurationModel",
+    "ServiceConfiguration",
+    "ConfigurationProvider",
+    "EnvProvider",
+    "FileProvider",
+]

processpype-0.1.0/processpype/core/configuration/manager.py

@@ -0,0 +1,126 @@
+"""Configuration manager for ProcessPype."""
+
+import asyncio
+from typing import Any
+
+from processpype.core.configuration.models import ApplicationConfiguration
+
+from .providers import ConfigurationProvider, EnvProvider, FileProvider
+
+
+class ConfigurationManager:
+    """Configuration manager for ProcessPype."""
+
+    def __init__(self) -> None:
+        """Initialize manager."""
+        self._providers: list[ConfigurationProvider] = []
+        self._config: dict[str, Any] = {}
+        self._initialized = False
+        self._lock = asyncio.Lock()
+
+    @classmethod
+    async def load_application_config(
+        cls, config_file: str | None = None, **kwargs: Any
+    ) -> ApplicationConfiguration:
+        """Load application configuration from file and/or kwargs.
+
+        Args:
+            config_file: Optional path to configuration file
+            **kwargs: Configuration overrides
+
+        Returns:
+            ApplicationConfiguration instance
+        """
+        # Create base configuration from kwargs
+        config = ApplicationConfiguration(**kwargs)
+
+        # If no config file, return the kwargs-based config
+        if not config_file:
+            return config
+
+        # Create manager and setup providers
+        manager = cls()
+        if config_file:
+            await manager.add_provider(FileProvider(config_file))
+        await manager.add_provider(EnvProvider())
+
+        # Initialize and load configuration
+        await manager.initialize()
+
+        # Return loaded configuration
+        return manager.get_model(ApplicationConfiguration)
+
+    def has_providers(self) -> bool:
+        """Check if there are any providers registered.
+
+        Returns:
+            True if there are providers registered, False otherwise
+        """
+        return len(self._providers) > 0
+
+    async def add_provider(self, provider: ConfigurationProvider) -> None:
+        """Add configuration provider.
+
+        Args:
+            provider: Configuration provider
+        """
+        async with self._lock:
+            self._providers.append(provider)
+            if self._initialized:
+                # Load configuration from the new provider
+                provider_config = await provider.load()
+                self._config.update(provider_config)
+
+    async def initialize(self) -> None:
+        """Initialize configuration manager."""
+        if self._initialized:
+            return
+
+        async with self._lock:
+            if not self._initialized:  # Double-check inside lock
+                # Load configuration from all providers in reverse order
+                config: dict[str, Any] = {}
+                for provider in reversed(self._providers):
+                    provider_config = await provider.load()
+                    config.update(provider_config)
+                self._config = config
+                self._initialized = True
+
+    def get(self, key: str, default: Any = None) -> Any:
+        """Get configuration value.
+
+        Args:
+            key: Configuration key
+            default: Default value if key not found
+
+        Returns:
+            Configuration value
+        """
+        return self._config.get(key, default)
+
+    def get_model(
+        self, model: type[ApplicationConfiguration]
+    ) -> ApplicationConfiguration:
+        """Get configuration as model.
+
+        Args:
+            model: Configuration model class
+
+        Returns:
+            Configuration model instance
+        """
+        return model.model_validate(self._config)
+
+    async def set(self, key: str, value: Any, save: bool = True) -> None:
+        """Set configuration value.
+
+        Args:
+            key: Configuration key
+            value: Configuration value
+            save: Whether to save to providers
+        """
+        async with self._lock:
+            self._config[key] = value
+            if save:
+                for provider in self._providers:
+                    await provider.save(self._config)