speedy-utils 1.1.43__tar.gz → 1.1.44__tar.gz

{speedy_utils-1.1.43 → speedy_utils-1.1.44}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: speedy-utils
-Version: 1.1.43
+Version: 1.1.44
 Summary: Fast and easy-to-use package for data science
 Project-URL: Homepage, https://github.com/anhvth/speedy
 Project-URL: Repository, https://github.com/anhvth/speedy
@@ -58,13 +58,48 @@ Description-Content-Type: text/markdown
 
 **Speedy Utils** is a Python utility library designed to streamline common programming tasks such as caching, parallel processing, file I/O, and data manipulation. It provides a collection of decorators, functions, and classes to enhance productivity and performance in your Python projects.
 
+## 🚀 Recent Updates (January 27, 2026)
+
+**Enhanced Error Handling in Parallel Processing:**
+- Rich-formatted error tracebacks with code context and syntax highlighting
+- Three error handling modes: 'raise', 'ignore', and 'log'
+- Filtered tracebacks focusing on user code (hiding infrastructure)
+- Real-time progress reporting with error/success statistics
+- Automatic error logging to timestamped files
+- Caller frame information showing where parallel functions were invoked
+
+## Quick Start
+
+### Parallel Processing with Error Handling
+
+```python
+from speedy_utils import multi_thread, multi_process
+
+# Simple parallel processing
+results = multi_thread(lambda x: x * 2, [1, 2, 3, 4, 5])
+# Results: [2, 4, 6, 8, 10]
+
+# Robust processing with error handling
+def process_item(item):
+    if item == 3:
+        raise ValueError(f"Cannot process item {item}")
+    return item * 2
+
+# Continue processing despite errors
+results = multi_thread(process_item, [1, 2, 3, 4, 5], error_handler='log')
+# Results: [2, 4, None, 8, 10] - errors logged automatically
+```
+
 ## Table of Contents
 
+- [🚀 Recent Updates](#-recent-updates-january-27-2026)
+- [Quick Start](#quick-start)
 - [Features](#features)
 - [Installation](#installation)
 - [Usage](#usage)
-  - [Caching](#caching)
   - [Parallel Processing](#parallel-processing)
+  - [Enhanced Error Handling](#enhanced-error-handling)
+  - [Caching](#caching)
   - [File I/O](#file-io)
   - [Data Manipulation](#data-manipulation)
   - [Utility Functions](#utility-functions)
@@ -73,11 +108,12 @@ Description-Content-Type: text/markdown
 ## Features
 
 - **Caching Mechanisms**: Disk-based and in-memory caching to optimize function calls.
-- **Parallel Processing**: Multi-threading, multi-processing, and asynchronous multi-threading utilities.
+- **Parallel Processing**: Multi-threading, multi-processing, and asynchronous multi-threading utilities with enhanced error handling.
 - **File I/O**: Simplified JSON, JSONL, and pickle file handling with support for various file extensions.
 - **Data Manipulation**: Utilities for flattening lists and dictionaries, converting data types, and more.
 - **Timing Utilities**: Tools to measure and log execution time of functions and processes.
 - **Pretty Printing**: Enhanced printing functions for structured data, including HTML tables for Jupyter notebooks.
+- **Enhanced Error Handling**: Rich error tracebacks with code context, configurable error handling modes ('raise', 'ignore', 'log'), and detailed progress reporting.
 
 ## Installation
 
@@ -162,20 +198,132 @@ result = compute_sum(5, 7)  # Retrieved from in-memory cache
 
 ### Parallel Processing
 
-#### Multi-threading
+#### Multi-threading with Enhanced Error Handling
 
-Execute functions concurrently using multiple threads. This approach is straightforward and automatically handles both notebook and Python script executions. In a notebook environment, it delegates the running thread to a separate process. If interrupted, it immediately stops this process, avoiding thread dependency issues where threads continue running until all tasks are completed.
+Execute functions concurrently using multiple threads with comprehensive error handling. The enhanced error handling provides three modes: 'raise' (default), 'ignore', and 'log'. When errors occur, you'll see rich-formatted tracebacks with code context and caller information.
 
 ```python
 from speedy_utils import multi_thread
 
 def process_item(item):
-    # Your processing logic
+    # Simulate processing that might fail
+    if item == 3:
+        raise ValueError(f"Invalid item: {item}")
     return item * 2
 
 items = [1, 2, 3, 4, 5]
-results = multi_thread(process_item, items, workers=3)
-print(results)  # [2, 4, 6, 8, 10]
+
+# Default behavior: raise on first error with rich traceback
+try:
+    results = multi_thread(process_item, items, workers=3)
+except SystemExit:
+    print("Error occurred and was displayed with rich formatting")
+
+# Continue processing on errors, return None for failed items
+results = multi_thread(process_item, items, workers=3, error_handler='ignore')
+print(results)  # [2, 4, None, 8, 10]
+
+# Log errors to files and continue processing
+results = multi_thread(process_item, items, workers=3, error_handler='log', max_error_files=10)
+print(results)  # [2, 4, None, 8, 10] - errors logged to .cache/speedy_utils/error_logs/
+```
+
+#### Multi-processing with Error Handling
+
+Process items across multiple processes with the same enhanced error handling capabilities.
+
+```python
+from speedy_utils import multi_process
+
+def risky_computation(x):
+    """Computation that might fail for certain inputs."""
+    if x % 5 == 0:
+        raise RuntimeError(f"Cannot process multiples of 5: {x}")
+    return x ** 2
+
+data = list(range(12))
+
+# Process with error logging (continues on errors)
+results = multi_process(
+    risky_computation, 
+    data, 
+    backend='mp',
+    error_handler='log',
+    max_error_files=5
+)
+print(results)  # [0, 1, 4, 9, 16, None, 36, 49, 64, 81, None, 121]
+```
+
+### Enhanced Error Handling
+
+**Speedy Utils** now provides comprehensive error handling for parallel processing with rich formatting and detailed diagnostics.
+
+#### Rich Error Tracebacks
+
+When errors occur, you'll see beautifully formatted tracebacks with:
+- **Code context**: Lines of code around the error location
+- **Caller information**: Shows where the parallel function was invoked
+- **Filtered frames**: Focuses on user code, hiding infrastructure details
+- **Color coding**: Easy-to-read formatting with syntax highlighting
+
+#### Error Handling Modes
+
+Choose how to handle errors in parallel processing:
+
+- **`'raise'` (default)**: Stop on first error with detailed traceback
+- **`'ignore'`**: Continue processing, return `None` for failed items  
+- **`'log'`**: Log errors to files and continue processing
+
+#### Error Logging
+
+When using `error_handler='log'`, errors are automatically saved to timestamped files in `.cache/speedy_utils/error_logs/` with full context and stack traces.
+
+#### Progress Reporting with Error Statistics
+
+Progress bars now show real-time error and success counts:
+
+```
+Multi-thread [8/10] [00:02<00:00, 3.45it/s, success=8, errors=2, pending=0]
+```
+
+This makes it easy to monitor processing health at a glance.
+
+#### Example: Robust Data Processing
+
+```python
+from speedy_utils import multi_thread
+
+def process_data_record(record):
+    """Process a data record that might have issues."""
+    try:
+        # Your processing logic here
+        value = record['value'] / record['divisor']
+        return {'result': value, 'status': 'success'}
+    except KeyError as e:
+        raise ValueError(f"Missing required field in record: {e}")
+    except ZeroDivisionError:
+        raise ValueError("Division by zero in record")
+
+# Sample data with some problematic records
+data = [
+    {'value': 10, 'divisor': 2},     # OK
+    {'value': 15, 'divisor': 0},     # Will error
+    {'value': 20, 'divisor': 4},     # OK
+    {'value': 25},                   # Missing divisor - will error
+]
+
+# Process with error logging - continues despite errors
+results = multi_thread(
+    process_data_record, 
+    data, 
+    workers=4,
+    error_handler='log',
+    max_error_files=10
+)
+
+print("Results:", results)
+# Output: Results: [{'result': 5.0, 'status': 'success'}, None, {'result': 5.0, 'status': 'success'}, None]
+# Errors are logged to files for later analysis
 ```
 
 ### File I/O

{speedy_utils-1.1.43 → speedy_utils-1.1.44}/README.md

@@ -6,13 +6,48 @@
 
 **Speedy Utils** is a Python utility library designed to streamline common programming tasks such as caching, parallel processing, file I/O, and data manipulation. It provides a collection of decorators, functions, and classes to enhance productivity and performance in your Python projects.
 
+## 🚀 Recent Updates (January 27, 2026)
+
+**Enhanced Error Handling in Parallel Processing:**
+- Rich-formatted error tracebacks with code context and syntax highlighting
+- Three error handling modes: 'raise', 'ignore', and 'log'
+- Filtered tracebacks focusing on user code (hiding infrastructure)
+- Real-time progress reporting with error/success statistics
+- Automatic error logging to timestamped files
+- Caller frame information showing where parallel functions were invoked
+
+## Quick Start
+
+### Parallel Processing with Error Handling
+
+```python
+from speedy_utils import multi_thread, multi_process
+
+# Simple parallel processing
+results = multi_thread(lambda x: x * 2, [1, 2, 3, 4, 5])
+# Results: [2, 4, 6, 8, 10]
+
+# Robust processing with error handling
+def process_item(item):
+    if item == 3:
+        raise ValueError(f"Cannot process item {item}")
+    return item * 2
+
+# Continue processing despite errors
+results = multi_thread(process_item, [1, 2, 3, 4, 5], error_handler='log')
+# Results: [2, 4, None, 8, 10] - errors logged automatically
+```
+
 ## Table of Contents
 
+- [🚀 Recent Updates](#-recent-updates-january-27-2026)
+- [Quick Start](#quick-start)
 - [Features](#features)
 - [Installation](#installation)
 - [Usage](#usage)
-  - [Caching](#caching)
   - [Parallel Processing](#parallel-processing)
+  - [Enhanced Error Handling](#enhanced-error-handling)
+  - [Caching](#caching)
   - [File I/O](#file-io)
   - [Data Manipulation](#data-manipulation)
   - [Utility Functions](#utility-functions)
@@ -21,11 +56,12 @@
 ## Features
 
 - **Caching Mechanisms**: Disk-based and in-memory caching to optimize function calls.
-- **Parallel Processing**: Multi-threading, multi-processing, and asynchronous multi-threading utilities.
+- **Parallel Processing**: Multi-threading, multi-processing, and asynchronous multi-threading utilities with enhanced error handling.
 - **File I/O**: Simplified JSON, JSONL, and pickle file handling with support for various file extensions.
 - **Data Manipulation**: Utilities for flattening lists and dictionaries, converting data types, and more.
 - **Timing Utilities**: Tools to measure and log execution time of functions and processes.
 - **Pretty Printing**: Enhanced printing functions for structured data, including HTML tables for Jupyter notebooks.
+- **Enhanced Error Handling**: Rich error tracebacks with code context, configurable error handling modes ('raise', 'ignore', 'log'), and detailed progress reporting.
 
 ## Installation
 
@@ -110,20 +146,132 @@ result = compute_sum(5, 7)  # Retrieved from in-memory cache
 
 ### Parallel Processing
 
-#### Multi-threading
+#### Multi-threading with Enhanced Error Handling
 
-Execute functions concurrently using multiple threads. This approach is straightforward and automatically handles both notebook and Python script executions. In a notebook environment, it delegates the running thread to a separate process. If interrupted, it immediately stops this process, avoiding thread dependency issues where threads continue running until all tasks are completed.
+Execute functions concurrently using multiple threads with comprehensive error handling. The enhanced error handling provides three modes: 'raise' (default), 'ignore', and 'log'. When errors occur, you'll see rich-formatted tracebacks with code context and caller information.
 
 ```python
 from speedy_utils import multi_thread
 
 def process_item(item):
-    # Your processing logic
+    # Simulate processing that might fail
+    if item == 3:
+        raise ValueError(f"Invalid item: {item}")
     return item * 2
 
 items = [1, 2, 3, 4, 5]
-results = multi_thread(process_item, items, workers=3)
-print(results)  # [2, 4, 6, 8, 10]
+
+# Default behavior: raise on first error with rich traceback
+try:
+    results = multi_thread(process_item, items, workers=3)
+except SystemExit:
+    print("Error occurred and was displayed with rich formatting")
+
+# Continue processing on errors, return None for failed items
+results = multi_thread(process_item, items, workers=3, error_handler='ignore')
+print(results)  # [2, 4, None, 8, 10]
+
+# Log errors to files and continue processing
+results = multi_thread(process_item, items, workers=3, error_handler='log', max_error_files=10)
+print(results)  # [2, 4, None, 8, 10] - errors logged to .cache/speedy_utils/error_logs/
+```
+
+#### Multi-processing with Error Handling
+
+Process items across multiple processes with the same enhanced error handling capabilities.
+
+```python
+from speedy_utils import multi_process
+
+def risky_computation(x):
+    """Computation that might fail for certain inputs."""
+    if x % 5 == 0:
+        raise RuntimeError(f"Cannot process multiples of 5: {x}")
+    return x ** 2
+
+data = list(range(12))
+
+# Process with error logging (continues on errors)
+results = multi_process(
+    risky_computation, 
+    data, 
+    backend='mp',
+    error_handler='log',
+    max_error_files=5
+)
+print(results)  # [0, 1, 4, 9, 16, None, 36, 49, 64, 81, None, 121]
+```
+
+### Enhanced Error Handling
+
+**Speedy Utils** now provides comprehensive error handling for parallel processing with rich formatting and detailed diagnostics.
+
+#### Rich Error Tracebacks
+
+When errors occur, you'll see beautifully formatted tracebacks with:
+- **Code context**: Lines of code around the error location
+- **Caller information**: Shows where the parallel function was invoked
+- **Filtered frames**: Focuses on user code, hiding infrastructure details
+- **Color coding**: Easy-to-read formatting with syntax highlighting
+
+#### Error Handling Modes
+
+Choose how to handle errors in parallel processing:
+
+- **`'raise'` (default)**: Stop on first error with detailed traceback
+- **`'ignore'`**: Continue processing, return `None` for failed items  
+- **`'log'`**: Log errors to files and continue processing
+
+#### Error Logging
+
+When using `error_handler='log'`, errors are automatically saved to timestamped files in `.cache/speedy_utils/error_logs/` with full context and stack traces.
+
+#### Progress Reporting with Error Statistics
+
+Progress bars now show real-time error and success counts:
+
+```
+Multi-thread [8/10] [00:02<00:00, 3.45it/s, success=8, errors=2, pending=0]
+```
+
+This makes it easy to monitor processing health at a glance.
+
+#### Example: Robust Data Processing
+
+```python
+from speedy_utils import multi_thread
+
+def process_data_record(record):
+    """Process a data record that might have issues."""
+    try:
+        # Your processing logic here
+        value = record['value'] / record['divisor']
+        return {'result': value, 'status': 'success'}
+    except KeyError as e:
+        raise ValueError(f"Missing required field in record: {e}")
+    except ZeroDivisionError:
+        raise ValueError("Division by zero in record")
+
+# Sample data with some problematic records
+data = [
+    {'value': 10, 'divisor': 2},     # OK
+    {'value': 15, 'divisor': 0},     # Will error
+    {'value': 20, 'divisor': 4},     # OK
+    {'value': 25},                   # Missing divisor - will error
+]
+
+# Process with error logging - continues despite errors
+results = multi_thread(
+    process_data_record, 
+    data, 
+    workers=4,
+    error_handler='log',
+    max_error_files=10
+)
+
+print("Results:", results)
+# Output: Results: [{'result': 5.0, 'status': 'success'}, None, {'result': 5.0, 'status': 'success'}, None]
+# Errors are logged to files for later analysis
 ```
 
 ### File I/O

{speedy_utils-1.1.43 → speedy_utils-1.1.44}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "speedy-utils"
-version = "1.1.43"
+version = "1.1.44"
 description = "Fast and easy-to-use package for data science"
 authors = [{ name = "AnhVTH", email = "anhvth.226@gmail.com" }]
 readme = "README.md"

{speedy_utils-1.1.43 → speedy_utils-1.1.44}/scripts/test_locals.py

@@ -15,5 +15,4 @@ def process_data(item):
 
 if __name__ == '__main__':
     inputs = range(5)
-    ret = multi_process(process_data, inputs, backend='mp', error_handler='log')
-    print(ret)
+    multi_process(process_data, inputs, backend='mp')

{speedy_utils-1.1.43 → speedy_utils-1.1.44}/scripts/test_ray_locals.py

@@ -3,8 +3,7 @@ from speedy_utils import *
 def do_something(x):
     x = 10
     y = 0
-    # Intentionally cause division by zero error for testing
-    _ = x/y
+    x/y
 
 if __name__ == '__main__':
     inputs = range(10)

{speedy_utils-1.1.43 → speedy_utils-1.1.44}/src/speedy_utils/multi_worker/process.py

@@ -115,32 +115,39 @@ class ErrorStats:
         func_name: str,
     ) -> str:
         """Write error details to a log file."""
+        from io import StringIO
+        from rich.console import Console
+        
         log_path = self._error_log_dir / f'{idx}.log'
         
+        output = StringIO()
+        console = Console(file=output, width=120, no_color=False)
+        
         # Format traceback
         tb_lines = self._format_traceback(error)
         
-        content = []
-        content.append(f'{"=" * 60}')
-        content.append(f'Error at index: {idx}')
-        content.append(f'Function: {func_name}')
-        content.append(f'Error Type: {type(error).__name__}')
-        content.append(f'Error Message: {error}')
-        content.append(f'{"=" * 60}')
-        content.append('')
-        content.append('Input:')
-        content.append('-' * 40)
+        console.print(f'{"=" * 60}')
+        console.print(f'Error at index: {idx}')
+        console.print(f'Function: {func_name}')
+        console.print(f'Error Type: {type(error).__name__}')
+        console.print(f'Error Message: {error}')
+        console.print(f'{"=" * 60}')
+        console.print('')
+        console.print('Input:')
+        console.print('-' * 40)
         try:
-            content.append(repr(input_value))
+            import json
+            console.print(json.dumps(input_value, indent=2))
         except Exception:
-            content.append('<unable to repr input>')
-        content.append('')
-        content.append('Traceback:')
-        content.append('-' * 40)
-        content.extend(tb_lines)
+            console.print(repr(input_value))
+        console.print('')
+        console.print('Traceback:')
+        console.print('-' * 40)
+        for line in tb_lines:
+            console.print(line)
         
         with open(log_path, 'w') as f:
-            f.write('\n'.join(content))
+            f.write(output.getvalue())
         
         return str(log_path)
 
@@ -824,7 +831,7 @@ def multi_process(
     log_worker: Literal['zero', 'first', 'all'] = 'first',
     total_items: int | None = None,
     poll_interval: float = 0.3,
-    error_handler: ErrorHandlerType = 'raise',
+    error_handler: ErrorHandlerType = 'log',
     max_error_files: int = 100,
     **func_kwargs: Any,
 ) -> list[Any]: