pyarallel 0.1.0__tar.gz → 0.1.1__tar.gz

{pyarallel-0.1.0 → pyarallel-0.1.1}/.gitignore

@@ -33,6 +33,7 @@ ENV/
 *.swo
 
 # Test
+.coverage.*
 .coverage
 htmlcov/
 .pytest_cache/

{pyarallel-0.1.0 → pyarallel-0.1.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pyarallel
-Version: 0.1.0
+Version: 0.1.1
 Summary: A powerful parallel execution library for Python
 Project-URL: Homepage, https://github.com/oneryalcin/pyarallel
 Project-URL: Repository, https://github.com/oneryalcin/pyarallel.git
@@ -280,4 +280,61 @@ Contributions are welcome! Please check out our [Contributing Guide](CONTRIBUTIN
 
 ## License
 
-This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+## Configuration
+
+Pyarallel provides a flexible configuration system that allows you to customize its behavior globally or per-function:
+
+### Basic Configuration
+
+```python
+from pyarallel import ConfigManager
+
+# Get the global configuration manager
+config = ConfigManager.get_instance()
+
+# Update configuration
+config.update_config({
+    "max_workers": 8,
+    "timeout": 60.0,
+    "debug": True
+})
+```
+
+### Configuration Options
+
+- **Execution Settings**
+  - `max_workers`: Maximum number of worker processes/threads (default: 4)
+  - `timeout`: Default timeout for parallel operations in seconds (default: 30.0)
+
+- **Resource Management**
+  - `memory_limit`: Memory limit per worker in bytes (default: None)
+  - `cpu_affinity`: Enable CPU affinity for workers (default: False)
+
+- **Logging and Debugging**
+  - `debug`: Enable debug mode (default: False)
+  - `log_level`: Logging level (default: "INFO")
+
+### Environment Variables
+
+You can configure Pyarallel using environment variables with the `PYARALLEL_` prefix:
+
+```bash
+PYARALLEL_MAX_WORKERS=4
+PYARALLEL_TIMEOUT=60.0
+PYARALLEL_DEBUG=true
+```
+
+### Configuration Files
+
+Load configuration from JSON, YAML, or TOML files:
+
+```python
+from pyarallel import PyarallelConfig
+
+# Load from file
+config = PyarallelConfig.from_file("pyarallel.yaml")
+
+# Convert to dictionary
+config_dict = config.to_dict()

{pyarallel-0.1.0 → pyarallel-0.1.1}/README.md

@@ -241,4 +241,61 @@ Contributions are welcome! Please check out our [Contributing Guide](CONTRIBUTIN
 
 ## License
 
-This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+## Configuration
+
+Pyarallel provides a flexible configuration system that allows you to customize its behavior globally or per-function:
+
+### Basic Configuration
+
+```python
+from pyarallel import ConfigManager
+
+# Get the global configuration manager
+config = ConfigManager.get_instance()
+
+# Update configuration
+config.update_config({
+    "max_workers": 8,
+    "timeout": 60.0,
+    "debug": True
+})
+```
+
+### Configuration Options
+
+- **Execution Settings**
+  - `max_workers`: Maximum number of worker processes/threads (default: 4)
+  - `timeout`: Default timeout for parallel operations in seconds (default: 30.0)
+
+- **Resource Management**
+  - `memory_limit`: Memory limit per worker in bytes (default: None)
+  - `cpu_affinity`: Enable CPU affinity for workers (default: False)
+
+- **Logging and Debugging**
+  - `debug`: Enable debug mode (default: False)
+  - `log_level`: Logging level (default: "INFO")
+
+### Environment Variables
+
+You can configure Pyarallel using environment variables with the `PYARALLEL_` prefix:
+
+```bash
+PYARALLEL_MAX_WORKERS=4
+PYARALLEL_TIMEOUT=60.0
+PYARALLEL_DEBUG=true
+```
+
+### Configuration Files
+
+Load configuration from JSON, YAML, or TOML files:
+
+```python
+from pyarallel import PyarallelConfig
+
+# Load from file
+config = PyarallelConfig.from_file("pyarallel.yaml")
+
+# Convert to dictionary
+config_dict = config.to_dict()

pyarallel-0.1.1/TASKS.md

@@ -0,0 +1,123 @@
+# Configuration System Implementation Tasks
+
+## 1. Core Configuration System [✅]
+- [✅] Create `config.py` with dataclass-based configuration
+- [✅] Implement singleton config manager
+- [✅] Add type validation for config values
+- [✅] Add merge strategy for partial updates
+
+**Test Cases:**
+```python
+def test_config_defaults():
+    """Default values are set correctly"""
+
+def test_config_validation():
+    """Invalid values raise proper exceptions"""
+
+def test_partial_update():
+    """Partial config updates don't affect other values"""
+```
+
+## 2. Environment Variables Support [✅]
+- [] Add env var parsing in config manager
+- [] Implement type coercion (str -> proper type)
+- [] Add prefix support (PYARALLEL_*)
+- [] Support complex values (lists, dicts via JSON)
+
+**Test Cases:**
+```python
+def test_env_var_loading():
+    """Config loads from environment variables"""
+
+def test_env_var_types():
+    """Environment variables are properly typed"""
+
+def test_env_var_prefix():
+    """Only PYARALLEL_* vars are loaded"""
+```
+
+## 3. Runtime Configuration API [✅]
+- [] Add global `set()` method
+- [] Add category-specific setters
+- [] Add value getters with dot notation
+- [] Implement config validation hooks
+
+**Test Cases:**
+```python
+def test_global_set():
+    """Global config can be set"""
+
+def test_category_set():
+    """Category-specific settings work"""
+
+def test_dot_notation():
+    """Dot notation access works for nested config"""
+```
+
+## 4. Decorator Integration [✅]
+- [] Update parallel decorator to use config
+- [] Add config override in decorator
+- [] Implement inheritance rules
+- [] Add runtime config warnings
+
+**Test Cases:**
+```python
+def test_decorator_defaults():
+    """Decorator uses global defaults"""
+
+def test_decorator_override():
+    """Decorator args override global config"""
+
+def test_runtime_warnings():
+    """Warnings for problematic configs"""
+```
+
+## 5. Documentation [ ]
+- [ ] Add configuration section to README
+- [ ] Document all environment variables
+- [ ] Add configuration examples
+- [ ] Document best practices
+
+## Configuration Schema
+```python
+{
+    "execution": {
+        "default_max_workers": int,
+        "default_executor_type": str,
+        "default_batch_size": Optional[int],
+        "prewarm_pools": bool
+    },
+    "rate_limiting": {
+        "default_rate": Optional[float],
+        "default_interval": str,
+        "burst_tolerance": float
+    },
+    "error_handling": {
+        "max_retries": int,
+        "retry_backoff": float,
+        "fail_fast": bool
+    },
+    "monitoring": {
+        "enable_logging": bool,
+        "log_level": str,
+        "sentry_dsn": Optional[str],
+        "metrics_enabled": bool
+    }
+}
+```
+
+## Environment Variables
+```
+PYARALLEL_MAX_WORKERS=4
+PYARALLEL_EXECUTOR_TYPE=thread
+PYARALLEL_BATCH_SIZE=10
+PYARALLEL_RATE_LIMIT=100/minute
+PYARALLEL_FAIL_FAST=true
+PYARALLEL_SENTRY_DSN=https://...
+```
+
+## Progress Tracking
+- ✅ Task completed
+- 🚧 In progress
+- ❌ Blocked
+- [ ] Not started

pyarallel-0.1.1/pyarallel/config.py

@@ -0,0 +1,110 @@
+"""Configuration module for pyarallel.
+
+This module provides a configuration system using Pydantic for schema validation,
+with support for loading configurations from different sources and thread-safe operations.
+"""
+
+from pathlib import Path
+from typing import Any, Dict, Optional, Union
+
+from pydantic import BaseModel, Field
+from pydantic_core import ValidationError
+
+
+class PyarallelConfig(BaseModel):
+    """Base configuration class for pyarallel.
+
+    This class defines the configuration schema and provides validation
+    using Pydantic. It includes default values and type hints for all settings.
+    """
+
+    # Execution settings
+    max_workers: int = Field(
+        default=4,
+        description="Maximum number of worker processes/threads",
+        ge=1
+    )
+    timeout: float = Field(
+        default=30.0,
+        description="Default timeout for parallel operations in seconds",
+        ge=0
+    )
+
+    # Resource management
+    memory_limit: Optional[int] = Field(
+        default=None,
+        description="Memory limit per worker in bytes, None for no limit"
+    )
+    cpu_affinity: bool = Field(
+        default=False,
+        description="Enable CPU affinity for workers"
+    )
+
+    # Logging and debugging
+    debug: bool = Field(
+        default=False,
+        description="Enable debug mode"
+    )
+    log_level: str = Field(
+        default="INFO",
+        description="Logging level"
+    )
+
+    @classmethod
+    def from_dict(cls, config_dict: Dict[str, Any]) -> "PyarallelConfig":
+        """Create a configuration instance from a dictionary.
+
+        Args:
+            config_dict: Dictionary containing configuration values
+
+        Returns:
+            PyarallelConfig instance
+
+        Raises:
+            ValidationError: If the configuration is invalid
+        """
+        return cls(**config_dict)
+
+    @classmethod
+    def from_file(cls, config_path: Union[str, Path]) -> "PyarallelConfig":
+        """Load configuration from a file (JSON, YAML, or TOML).
+
+        Args:
+            config_path: Path to the configuration file
+
+        Returns:
+            PyarallelConfig instance
+
+        Raises:
+            FileNotFoundError: If the configuration file doesn't exist
+            ValidationError: If the configuration is invalid
+        """
+        config_path = Path(config_path)
+        if not config_path.exists():
+            raise FileNotFoundError(f"Configuration file not found: {config_path}")
+
+        # Load based on file extension
+        if config_path.suffix == ".json":
+            import json
+            with open(config_path) as f:
+                config_dict = json.load(f)
+        elif config_path.suffix in (".yml", ".yaml"):
+            import yaml
+            with open(config_path) as f:
+                config_dict = yaml.safe_load(f)
+        elif config_path.suffix == ".toml":
+            import toml
+            with open(config_path) as f:
+                config_dict = toml.load(f)
+        else:
+            raise ValueError(f"Unsupported configuration file format: {config_path.suffix}")
+
+        return cls.from_dict(config_dict)
+
+    def to_dict(self) -> Dict[str, Any]:
+        """Convert configuration to a dictionary.
+
+        Returns:
+            Dictionary representation of the configuration
+        """
+        return self.model_dump()

pyarallel-0.1.1/pyarallel/config_manager.py

@@ -0,0 +1,90 @@
+"""Configuration manager module for pyarallel.
+
+This module provides a thread-safe singleton configuration manager that handles
+all configuration operations and maintains the global configuration state.
+"""
+
+from threading import Lock
+from typing import Any, Dict, Optional, Type, TypeVar
+
+from .config import PyarallelConfig
+
+T = TypeVar('T', bound='ConfigManager')
+
+
+class ConfigManager:
+    """Singleton configuration manager for pyarallel.
+
+    This class ensures thread-safe access to configuration settings and provides
+    methods for updating and retrieving configuration values.
+    """
+    _instance: Optional[T] = None
+    _lock: Lock = Lock()
+    _config: Optional[PyarallelConfig] = None
+
+    def __new__(cls: Type[T]) -> T:
+        if not cls._instance:
+            with cls._lock:
+                if not cls._instance:
+                    cls._instance = super().__new__(cls)
+                    cls._instance._config = PyarallelConfig()
+        return cls._instance
+
+    @classmethod
+    def get_instance(cls: Type[T]) -> T:
+        """Get the singleton instance of the configuration manager.
+
+        Returns:
+            ConfigManager: The singleton instance
+        """
+        return cls()
+
+    def get_config(self) -> PyarallelConfig:
+        """Get the current configuration.
+
+        Returns:
+            PyarallelConfig: The current configuration
+        """
+        return self._config
+
+    def update_config(self, updates: Dict[str, Any]) -> None:
+        """Update the configuration with new values.
+
+        This method implements a merge strategy that allows partial updates
+        while preserving existing values.
+
+        Args:
+            updates: Dictionary containing the configuration updates
+        """
+        with self._lock:
+            current_config = self._config.model_dump()
+            # Handle nested execution structure
+            if "execution" in updates:
+                for key, value in updates["execution"].items():
+                    updates[key] = value
+                del updates["execution"]
+            
+            # Validate max_workers before merging
+            if "max_workers" in updates and updates["max_workers"] < 1:
+                updates["max_workers"] = 1
+                
+            merged_config = self._deep_merge(current_config, updates)
+            self._config = PyarallelConfig.from_dict(merged_config)
+
+    def _deep_merge(self, base: Dict[str, Any], updates: Dict[str, Any]) -> Dict[str, Any]:
+        """Recursively merge two dictionaries.
+
+        Args:
+            base: The base dictionary
+            updates: The dictionary with updates
+
+        Returns:
+            Dict[str, Any]: The merged dictionary
+        """
+        result = base.copy()
+        for key, value in updates.items():
+            if key in result and isinstance(result[key], dict) and isinstance(value, dict):
+                result[key] = self._deep_merge(result[key], value)
+            else:
+                result[key] = value
+        return result

{pyarallel-0.1.0 → pyarallel-0.1.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "pyarallel"
-version = "0.1.0"
+version = "0.1.1"
 description = "A powerful parallel execution library for Python"
 readme = "README.md"
 requires-python = ">=3.12"

{pyarallel-0.1.0 → pyarallel-0.1.1}/setup.py

@@ -5,7 +5,7 @@ with open("README.md", "r", encoding="utf-8") as fh:
 
 setup(
     name="pyarallel",
-    version="0.1.0",
+    version="0.1.1",
     author="Mehmet Oner Yalcin",
     author_email="oneryalcin@gmail.com",
     description="A powerful parallel execution library for Python",

pyarallel-0.1.1/tests/test_config_manager.py

@@ -0,0 +1,69 @@
+"""Test cases for the configuration manager module.
+
+This module contains tests for the singleton pattern, thread safety,
+and merge strategy of the configuration manager.
+"""
+
+import threading
+from concurrent.futures import ThreadPoolExecutor
+
+import pytest
+
+from pyarallel.config_manager import ConfigManager
+
+
+def test_singleton_pattern():
+    """Test that ConfigManager maintains singleton pattern."""
+    config1 = ConfigManager()
+    config2 = ConfigManager()
+    assert config1 is config2
+
+
+def test_thread_safety():
+    """Test thread-safe access to configuration."""
+    def update_config(i):
+        manager = ConfigManager()
+        manager.update_config({"max_workers": i})
+        return manager.get_config().max_workers
+
+    with ThreadPoolExecutor(max_workers=4) as executor:
+        results = list(executor.map(update_config, range(10)))
+    
+    # Verify the final state is consistent
+    assert ConfigManager().get_config().max_workers == 9
+
+
+def test_partial_update():
+    """Test that partial updates don't affect other values."""
+    manager = ConfigManager()
+    
+    # Initial state
+    initial_config = manager.get_config()
+    initial_timeout = initial_config.timeout
+    
+    # Update only max_workers
+    manager.update_config({"max_workers": 8})
+    
+    # Verify timeout remains unchanged
+    updated_config = manager.get_config()
+    assert updated_config.timeout == initial_timeout
+    assert updated_config.max_workers == 8
+
+
+def test_nested_merge():
+    """Test deep merging of nested configuration values."""
+    manager = ConfigManager()
+    
+    # Update with nested structure
+    update = {
+        "execution": {
+            "max_workers": 8,
+            "timeout": 60.0
+        }
+    }
+    
+    manager.update_config(update)
+    config = manager.get_config()
+    
+    assert config.max_workers == 8
+    assert config.timeout == 60.0

pyarallel-0.1.1/uv.lock

@@ -0,0 +1,7 @@
+version = 1
+requires-python = ">=3.12"
+
+[[package]]
+name = "pyarallel"
+version = "0.1.0"
+source = { editable = "." }

pyarallel-0.1.0/hello.py

@@ -1,6 +0,0 @@
-def main():
-    print("Hello from pyarallel!")
-
-
-if __name__ == "__main__":
-    main()