speedy-utils 1.1.42__tar.gz → 1.1.44__tar.gz

speedy_utils-1.1.44/.github/prompts/improveParallelErrorHandling.prompt.md

@@ -0,0 +1,64 @@
+---
+name: improveParallelErrorHandling
+description: Enhance error tracebacks in parallel execution with rich formatting and context
+argument-hint: the parallel execution function and backend type
+---
+
+Improve error handling for the specified parallel execution function to provide clean, user-focused tracebacks similar to direct function calls.
+
+## Requirements
+
+1. **Filter Internal Frames**: Remove framework/library internal frames from tracebacks, showing only user code
+2. **Add Context Lines**: Display 3 lines before and after each error location with line numbers
+3. **Include Caller Frame**: Show where the parallel execution function was called, not just where the error occurred
+4. **Rich Formatting**: Use rich library's Panel/formatting for clean, readable output
+5. **Suppress Noise**: Set environment variables or flags to suppress verbose framework error logs
+
+## Implementation Steps
+
+1. **Capture Caller Context**: Use `inspect.currentframe().f_back` to capture where the parallel function was called (filename, line number, function name)
+
+2. **Wrap Error Handling**: Catch framework-specific exceptions (e.g., `RayTaskError`, thread exceptions) in the execution loop
+
+3. **Parse/Extract Original Exception**: Get the underlying user exception from the framework wrapper
+   - Extract exception type, message, and traceback information
+   - Parse from string representation if traceback objects aren't preserved
+
+4. **Filter Frames**: Skip frames matching internal paths:
+   - Framework internals (e.g., `ray/_private`, `concurrent/futures`)
+   - Library worker implementations (e.g., `speedy_utils/multi_worker`)
+   - Site-packages for the framework
+
+5. **Format with Context**:
+   - For each user frame, show: `filepath:lineno in function_name`
+   - Use `linecache.getline()` to retrieve surrounding lines
+   - Highlight the error line with `❱` marker
+   - Number all lines (e.g., `   4 │ code here` or `   5 ❱ error here`)
+
+6. **Display Caller Frame First**: Show where the parallel function was invoked before showing the actual error location
+
+7. **Clean Exit**: Flush output streams before exiting to ensure traceback displays
+
+## Example Output Format
+
+```
+╭─────────────── Traceback (most recent call last) ───────────────╮
+│ /path/to/user/script.py:42 in main                              │
+│                                                                  │
+│   40 │ data = load_data()                                        │
+│   41 │ # Process in parallel                                     │
+│   42 ❱ results = multi_process(process_item, data, workers=8)   │
+│   43 │                                                           │
+│                                                                  │
+│ /path/to/user/module.py:15 in process_item                      │
+│                                                                  │
+│   12 │ def process_item(item):                                   │
+│   13 │     value = item['key']                                   │
+│   14 │     denominator = value - 100                             │
+│   15 ❱     return 1 / denominator                                │
+│   16 │                                                           │
+╰──────────────────────────────────────────────────────────────────╯
+ZeroDivisionError: division by zero
+```
+
+Apply these improvements to the specified parallel execution function, ensuring error messages are as clear as direct function calls while maintaining all performance benefits of parallel execution.

{speedy_utils-1.1.42 → speedy_utils-1.1.44}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: speedy-utils
-Version: 1.1.42
+Version: 1.1.44
 Summary: Fast and easy-to-use package for data science
 Project-URL: Homepage, https://github.com/anhvth/speedy
 Project-URL: Repository, https://github.com/anhvth/speedy
@@ -17,7 +17,7 @@ Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
-Requires-Python: >=3.8
+Requires-Python: >=3.9
 Requires-Dist: aiohttp
 Requires-Dist: bump2version
 Requires-Dist: cachetools
@@ -39,6 +39,7 @@ Requires-Dist: pydantic
 Requires-Dist: pytest
 Requires-Dist: ray
 Requires-Dist: requests
+Requires-Dist: rich>=14.3.1
 Requires-Dist: ruff
 Requires-Dist: scikit-learn
 Requires-Dist: tabulate
@@ -57,13 +58,48 @@ Description-Content-Type: text/markdown
 
 **Speedy Utils** is a Python utility library designed to streamline common programming tasks such as caching, parallel processing, file I/O, and data manipulation. It provides a collection of decorators, functions, and classes to enhance productivity and performance in your Python projects.
 
+## 🚀 Recent Updates (January 27, 2026)
+
+**Enhanced Error Handling in Parallel Processing:**
+- Rich-formatted error tracebacks with code context and syntax highlighting
+- Three error handling modes: 'raise', 'ignore', and 'log'
+- Filtered tracebacks focusing on user code (hiding infrastructure)
+- Real-time progress reporting with error/success statistics
+- Automatic error logging to timestamped files
+- Caller frame information showing where parallel functions were invoked
+
+## Quick Start
+
+### Parallel Processing with Error Handling
+
+```python
+from speedy_utils import multi_thread, multi_process
+
+# Simple parallel processing
+results = multi_thread(lambda x: x * 2, [1, 2, 3, 4, 5])
+# Results: [2, 4, 6, 8, 10]
+
+# Robust processing with error handling
+def process_item(item):
+    if item == 3:
+        raise ValueError(f"Cannot process item {item}")
+    return item * 2
+
+# Continue processing despite errors
+results = multi_thread(process_item, [1, 2, 3, 4, 5], error_handler='log')
+# Results: [2, 4, None, 8, 10] - errors logged automatically
+```
+
 ## Table of Contents
 
+- [🚀 Recent Updates](#-recent-updates-january-27-2026)
+- [Quick Start](#quick-start)
 - [Features](#features)
 - [Installation](#installation)
 - [Usage](#usage)
-  - [Caching](#caching)
   - [Parallel Processing](#parallel-processing)
+  - [Enhanced Error Handling](#enhanced-error-handling)
+  - [Caching](#caching)
   - [File I/O](#file-io)
   - [Data Manipulation](#data-manipulation)
   - [Utility Functions](#utility-functions)
@@ -72,11 +108,12 @@ Description-Content-Type: text/markdown
 ## Features
 
 - **Caching Mechanisms**: Disk-based and in-memory caching to optimize function calls.
-- **Parallel Processing**: Multi-threading, multi-processing, and asynchronous multi-threading utilities.
+- **Parallel Processing**: Multi-threading, multi-processing, and asynchronous multi-threading utilities with enhanced error handling.
 - **File I/O**: Simplified JSON, JSONL, and pickle file handling with support for various file extensions.
 - **Data Manipulation**: Utilities for flattening lists and dictionaries, converting data types, and more.
 - **Timing Utilities**: Tools to measure and log execution time of functions and processes.
 - **Pretty Printing**: Enhanced printing functions for structured data, including HTML tables for Jupyter notebooks.
+- **Enhanced Error Handling**: Rich error tracebacks with code context, configurable error handling modes ('raise', 'ignore', 'log'), and detailed progress reporting.
 
 ## Installation
 
@@ -161,20 +198,132 @@ result = compute_sum(5, 7)  # Retrieved from in-memory cache
 
 ### Parallel Processing
 
-#### Multi-threading
+#### Multi-threading with Enhanced Error Handling
 
-Execute functions concurrently using multiple threads. This approach is straightforward and automatically handles both notebook and Python script executions. In a notebook environment, it delegates the running thread to a separate process. If interrupted, it immediately stops this process, avoiding thread dependency issues where threads continue running until all tasks are completed.
+Execute functions concurrently using multiple threads with comprehensive error handling. The enhanced error handling provides three modes: 'raise' (default), 'ignore', and 'log'. When errors occur, you'll see rich-formatted tracebacks with code context and caller information.
 
 ```python
 from speedy_utils import multi_thread
 
 def process_item(item):
-    # Your processing logic
+    # Simulate processing that might fail
+    if item == 3:
+        raise ValueError(f"Invalid item: {item}")
     return item * 2
 
 items = [1, 2, 3, 4, 5]
-results = multi_thread(process_item, items, workers=3)
-print(results)  # [2, 4, 6, 8, 10]
+
+# Default behavior: raise on first error with rich traceback
+try:
+    results = multi_thread(process_item, items, workers=3)
+except SystemExit:
+    print("Error occurred and was displayed with rich formatting")
+
+# Continue processing on errors, return None for failed items
+results = multi_thread(process_item, items, workers=3, error_handler='ignore')
+print(results)  # [2, 4, None, 8, 10]
+
+# Log errors to files and continue processing
+results = multi_thread(process_item, items, workers=3, error_handler='log', max_error_files=10)
+print(results)  # [2, 4, None, 8, 10] - errors logged to .cache/speedy_utils/error_logs/
+```
+
+#### Multi-processing with Error Handling
+
+Process items across multiple processes with the same enhanced error handling capabilities.
+
+```python
+from speedy_utils import multi_process
+
+def risky_computation(x):
+    """Computation that might fail for certain inputs."""
+    if x % 5 == 0:
+        raise RuntimeError(f"Cannot process multiples of 5: {x}")
+    return x ** 2
+
+data = list(range(12))
+
+# Process with error logging (continues on errors)
+results = multi_process(
+    risky_computation, 
+    data, 
+    backend='mp',
+    error_handler='log',
+    max_error_files=5
+)
+print(results)  # [0, 1, 4, 9, 16, None, 36, 49, 64, 81, None, 121]
+```
+
+### Enhanced Error Handling
+
+**Speedy Utils** now provides comprehensive error handling for parallel processing with rich formatting and detailed diagnostics.
+
+#### Rich Error Tracebacks
+
+When errors occur, you'll see beautifully formatted tracebacks with:
+- **Code context**: Lines of code around the error location
+- **Caller information**: Shows where the parallel function was invoked
+- **Filtered frames**: Focuses on user code, hiding infrastructure details
+- **Color coding**: Easy-to-read formatting with syntax highlighting
+
+#### Error Handling Modes
+
+Choose how to handle errors in parallel processing:
+
+- **`'raise'` (default)**: Stop on first error with detailed traceback
+- **`'ignore'`**: Continue processing, return `None` for failed items  
+- **`'log'`**: Log errors to files and continue processing
+
+#### Error Logging
+
+When using `error_handler='log'`, errors are automatically saved to timestamped files in `.cache/speedy_utils/error_logs/` with full context and stack traces.
+
+#### Progress Reporting with Error Statistics
+
+Progress bars now show real-time error and success counts:
+
+```
+Multi-thread [8/10] [00:02<00:00, 3.45it/s, success=8, errors=2, pending=0]
+```
+
+This makes it easy to monitor processing health at a glance.
+
+#### Example: Robust Data Processing
+
+```python
+from speedy_utils import multi_thread
+
+def process_data_record(record):
+    """Process a data record that might have issues."""
+    try:
+        # Your processing logic here
+        value = record['value'] / record['divisor']
+        return {'result': value, 'status': 'success'}
+    except KeyError as e:
+        raise ValueError(f"Missing required field in record: {e}")
+    except ZeroDivisionError:
+        raise ValueError("Division by zero in record")
+
+# Sample data with some problematic records
+data = [
+    {'value': 10, 'divisor': 2},     # OK
+    {'value': 15, 'divisor': 0},     # Will error
+    {'value': 20, 'divisor': 4},     # OK
+    {'value': 25},                   # Missing divisor - will error
+]
+
+# Process with error logging - continues despite errors
+results = multi_thread(
+    process_data_record, 
+    data, 
+    workers=4,
+    error_handler='log',
+    max_error_files=10
+)
+
+print("Results:", results)
+# Output: Results: [{'result': 5.0, 'status': 'success'}, None, {'result': 5.0, 'status': 'success'}, None]
+# Errors are logged to files for later analysis
 ```
 
 ### File I/O

{speedy_utils-1.1.42 → speedy_utils-1.1.44}/README.md

@@ -6,13 +6,48 @@
 
 **Speedy Utils** is a Python utility library designed to streamline common programming tasks such as caching, parallel processing, file I/O, and data manipulation. It provides a collection of decorators, functions, and classes to enhance productivity and performance in your Python projects.
 
+## 🚀 Recent Updates (January 27, 2026)
+
+**Enhanced Error Handling in Parallel Processing:**
+- Rich-formatted error tracebacks with code context and syntax highlighting
+- Three error handling modes: 'raise', 'ignore', and 'log'
+- Filtered tracebacks focusing on user code (hiding infrastructure)
+- Real-time progress reporting with error/success statistics
+- Automatic error logging to timestamped files
+- Caller frame information showing where parallel functions were invoked
+
+## Quick Start
+
+### Parallel Processing with Error Handling
+
+```python
+from speedy_utils import multi_thread, multi_process
+
+# Simple parallel processing
+results = multi_thread(lambda x: x * 2, [1, 2, 3, 4, 5])
+# Results: [2, 4, 6, 8, 10]
+
+# Robust processing with error handling
+def process_item(item):
+    if item == 3:
+        raise ValueError(f"Cannot process item {item}")
+    return item * 2
+
+# Continue processing despite errors
+results = multi_thread(process_item, [1, 2, 3, 4, 5], error_handler='log')
+# Results: [2, 4, None, 8, 10] - errors logged automatically
+```
+
 ## Table of Contents
 
+- [🚀 Recent Updates](#-recent-updates-january-27-2026)
+- [Quick Start](#quick-start)
 - [Features](#features)
 - [Installation](#installation)
 - [Usage](#usage)
-  - [Caching](#caching)
   - [Parallel Processing](#parallel-processing)
+  - [Enhanced Error Handling](#enhanced-error-handling)
+  - [Caching](#caching)
   - [File I/O](#file-io)
   - [Data Manipulation](#data-manipulation)
   - [Utility Functions](#utility-functions)
@@ -21,11 +56,12 @@
 ## Features
 
 - **Caching Mechanisms**: Disk-based and in-memory caching to optimize function calls.
-- **Parallel Processing**: Multi-threading, multi-processing, and asynchronous multi-threading utilities.
+- **Parallel Processing**: Multi-threading, multi-processing, and asynchronous multi-threading utilities with enhanced error handling.
 - **File I/O**: Simplified JSON, JSONL, and pickle file handling with support for various file extensions.
 - **Data Manipulation**: Utilities for flattening lists and dictionaries, converting data types, and more.
 - **Timing Utilities**: Tools to measure and log execution time of functions and processes.
 - **Pretty Printing**: Enhanced printing functions for structured data, including HTML tables for Jupyter notebooks.
+- **Enhanced Error Handling**: Rich error tracebacks with code context, configurable error handling modes ('raise', 'ignore', 'log'), and detailed progress reporting.
 
 ## Installation
 
@@ -110,20 +146,132 @@ result = compute_sum(5, 7)  # Retrieved from in-memory cache
 
 ### Parallel Processing
 
-#### Multi-threading
+#### Multi-threading with Enhanced Error Handling
 
-Execute functions concurrently using multiple threads. This approach is straightforward and automatically handles both notebook and Python script executions. In a notebook environment, it delegates the running thread to a separate process. If interrupted, it immediately stops this process, avoiding thread dependency issues where threads continue running until all tasks are completed.
+Execute functions concurrently using multiple threads with comprehensive error handling. The enhanced error handling provides three modes: 'raise' (default), 'ignore', and 'log'. When errors occur, you'll see rich-formatted tracebacks with code context and caller information.
 
 ```python
 from speedy_utils import multi_thread
 
 def process_item(item):
-    # Your processing logic
+    # Simulate processing that might fail
+    if item == 3:
+        raise ValueError(f"Invalid item: {item}")
     return item * 2
 
 items = [1, 2, 3, 4, 5]
-results = multi_thread(process_item, items, workers=3)
-print(results)  # [2, 4, 6, 8, 10]
+
+# Default behavior: raise on first error with rich traceback
+try:
+    results = multi_thread(process_item, items, workers=3)
+except SystemExit:
+    print("Error occurred and was displayed with rich formatting")
+
+# Continue processing on errors, return None for failed items
+results = multi_thread(process_item, items, workers=3, error_handler='ignore')
+print(results)  # [2, 4, None, 8, 10]
+
+# Log errors to files and continue processing
+results = multi_thread(process_item, items, workers=3, error_handler='log', max_error_files=10)
+print(results)  # [2, 4, None, 8, 10] - errors logged to .cache/speedy_utils/error_logs/
+```
+
+#### Multi-processing with Error Handling
+
+Process items across multiple processes with the same enhanced error handling capabilities.
+
+```python
+from speedy_utils import multi_process
+
+def risky_computation(x):
+    """Computation that might fail for certain inputs."""
+    if x % 5 == 0:
+        raise RuntimeError(f"Cannot process multiples of 5: {x}")
+    return x ** 2
+
+data = list(range(12))
+
+# Process with error logging (continues on errors)
+results = multi_process(
+    risky_computation, 
+    data, 
+    backend='mp',
+    error_handler='log',
+    max_error_files=5
+)
+print(results)  # [0, 1, 4, 9, 16, None, 36, 49, 64, 81, None, 121]
+```
+
+### Enhanced Error Handling
+
+**Speedy Utils** now provides comprehensive error handling for parallel processing with rich formatting and detailed diagnostics.
+
+#### Rich Error Tracebacks
+
+When errors occur, you'll see beautifully formatted tracebacks with:
+- **Code context**: Lines of code around the error location
+- **Caller information**: Shows where the parallel function was invoked
+- **Filtered frames**: Focuses on user code, hiding infrastructure details
+- **Color coding**: Easy-to-read formatting with syntax highlighting
+
+#### Error Handling Modes
+
+Choose how to handle errors in parallel processing:
+
+- **`'raise'` (default)**: Stop on first error with detailed traceback
+- **`'ignore'`**: Continue processing, return `None` for failed items  
+- **`'log'`**: Log errors to files and continue processing
+
+#### Error Logging
+
+When using `error_handler='log'`, errors are automatically saved to timestamped files in `.cache/speedy_utils/error_logs/` with full context and stack traces.
+
+#### Progress Reporting with Error Statistics
+
+Progress bars now show real-time error and success counts:
+
+```
+Multi-thread [8/10] [00:02<00:00, 3.45it/s, success=8, errors=2, pending=0]
+```
+
+This makes it easy to monitor processing health at a glance.
+
+#### Example: Robust Data Processing
+
+```python
+from speedy_utils import multi_thread
+
+def process_data_record(record):
+    """Process a data record that might have issues."""
+    try:
+        # Your processing logic here
+        value = record['value'] / record['divisor']
+        return {'result': value, 'status': 'success'}
+    except KeyError as e:
+        raise ValueError(f"Missing required field in record: {e}")
+    except ZeroDivisionError:
+        raise ValueError("Division by zero in record")
+
+# Sample data with some problematic records
+data = [
+    {'value': 10, 'divisor': 2},     # OK
+    {'value': 15, 'divisor': 0},     # Will error
+    {'value': 20, 'divisor': 4},     # OK
+    {'value': 25},                   # Missing divisor - will error
+]
+
+# Process with error logging - continues despite errors
+results = multi_thread(
+    process_data_record, 
+    data, 
+    workers=4,
+    error_handler='log',
+    max_error_files=10
+)
+
+print("Results:", results)
+# Output: Results: [{'result': 5.0, 'status': 'success'}, None, {'result': 5.0, 'status': 'success'}, None]
+# Errors are logged to files for later analysis
 ```
 
 ### File I/O

{speedy_utils-1.1.42 → speedy_utils-1.1.44}/pyproject.toml

@@ -1,11 +1,11 @@
 [project]
 name = "speedy-utils"
-version = "1.1.42"
+version = "1.1.44"
 description = "Fast and easy-to-use package for data science"
 authors = [{ name = "AnhVTH", email = "anhvth.226@gmail.com" }]
 readme = "README.md"
 license = { text = "MIT" }
-requires-python = ">=3.8"
+requires-python = ">=3.9"
 dependencies = [
     "numpy",
     "requests",
@@ -33,6 +33,7 @@ dependencies = [
     "ray",
     "aiohttp",
     "pytest",
+    "rich>=14.3.1",
 ]
 classifiers = [
     "Development Status :: 4 - Beta",

speedy_utils-1.1.44/scripts/bug.py

@@ -0,0 +1,34 @@
+# type: ignore
+from speedy_utils import multi_process, multi_thread
+
+
+def do_something(x):
+    if x % 3 == 0:
+        raise ValueError(f'Error at index {x}')
+    return x * 2
+
+
+inputs = range(10)
+
+
+if __name__ == '__main__':
+    print('Testing error_handler="log" with mp backend:')
+    results = multi_process(
+        do_something,
+        inputs,
+        backend='mp',
+        error_handler='log',
+        max_error_files=5,
+    )
+    print(f'Results: {results}')
+    print()
+
+    # print('Testing error_handler="log" with multi_thread:')
+    # results = multi_thread(
+    #     do_something,
+    #     inputs,
+    #     error_handler='log',
+    #     max_error_files=5,
+    # )
+    # print(f'Results: {results}')
+

speedy_utils-1.1.44/scripts/bug_simple.py

@@ -0,0 +1,11 @@
+from speedy_utils import *
+
+
+def do_something(x):
+    x = 10
+    y = 0
+    x/y
+
+
+do_something(1)
+