speedy-utils 1.1.43__tar.gz → 1.1.45__tar.gz

{speedy_utils-1.1.43 → speedy_utils-1.1.45}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: speedy-utils
-Version: 1.1.43
+Version: 1.1.45
 Summary: Fast and easy-to-use package for data science
 Project-URL: Homepage, https://github.com/anhvth/speedy
 Project-URL: Repository, https://github.com/anhvth/speedy
@@ -58,13 +58,49 @@ Description-Content-Type: text/markdown
 
 **Speedy Utils** is a Python utility library designed to streamline common programming tasks such as caching, parallel processing, file I/O, and data manipulation. It provides a collection of decorators, functions, and classes to enhance productivity and performance in your Python projects.
 
+## 🚀 Recent Updates (January 27, 2026)
+
+**Enhanced Error Handling in Parallel Processing:**
+
+- Rich-formatted error tracebacks with code context and syntax highlighting
+- Three error handling modes: 'raise', 'ignore', and 'log'
+- Filtered tracebacks focusing on user code (hiding infrastructure)
+- Real-time progress reporting with error/success statistics
+- Automatic error logging to timestamped files
+- Caller frame information showing where parallel functions were invoked
+
+## Quick Start
+
+### Parallel Processing with Error Handling
+
+```python
+from speedy_utils import multi_thread, multi_process
+
+# Simple parallel processing
+results = multi_thread(lambda x: x * 2, [1, 2, 3, 4, 5])
+# Results: [2, 4, 6, 8, 10]
+
+# Robust processing with error handling
+def process_item(item):
+    if item == 3:
+        raise ValueError(f"Cannot process item {item}")
+    return item * 2
+
+# Continue processing despite errors
+results = multi_thread(process_item, [1, 2, 3, 4, 5], error_handler='log')
+# Results: [2, 4, None, 8, 10] - errors logged automatically
+```
+
 ## Table of Contents
 
+- [🚀 Recent Updates](#-recent-updates-january-27-2026)
+- [Quick Start](#quick-start)
 - [Features](#features)
 - [Installation](#installation)
 - [Usage](#usage)
-  - [Caching](#caching)
   - [Parallel Processing](#parallel-processing)
+  - [Enhanced Error Handling](#enhanced-error-handling)
+  - [Caching](#caching)
   - [File I/O](#file-io)
   - [Data Manipulation](#data-manipulation)
   - [Utility Functions](#utility-functions)
@@ -73,11 +109,12 @@ Description-Content-Type: text/markdown
 ## Features
 
 - **Caching Mechanisms**: Disk-based and in-memory caching to optimize function calls.
-- **Parallel Processing**: Multi-threading, multi-processing, and asynchronous multi-threading utilities.
+- **Parallel Processing**: Multi-threading, multi-processing, and asynchronous multi-threading utilities with enhanced error handling.
 - **File I/O**: Simplified JSON, JSONL, and pickle file handling with support for various file extensions.
 - **Data Manipulation**: Utilities for flattening lists and dictionaries, converting data types, and more.
 - **Timing Utilities**: Tools to measure and log execution time of functions and processes.
 - **Pretty Printing**: Enhanced printing functions for structured data, including HTML tables for Jupyter notebooks.
+- **Enhanced Error Handling**: Rich error tracebacks with code context, configurable error handling modes ('raise', 'ignore', 'log'), and detailed progress reporting.
 
 ## Installation
 
@@ -162,20 +199,133 @@ result = compute_sum(5, 7)  # Retrieved from in-memory cache
 
 ### Parallel Processing
 
-#### Multi-threading
+#### Multi-threading with Enhanced Error Handling
 
-Execute functions concurrently using multiple threads. This approach is straightforward and automatically handles both notebook and Python script executions. In a notebook environment, it delegates the running thread to a separate process. If interrupted, it immediately stops this process, avoiding thread dependency issues where threads continue running until all tasks are completed.
+Execute functions concurrently using multiple threads with comprehensive error handling. The enhanced error handling provides three modes: 'raise' (default), 'ignore', and 'log'. When errors occur, you'll see rich-formatted tracebacks with code context and caller information.
 
 ```python
 from speedy_utils import multi_thread
 
 def process_item(item):
-    # Your processing logic
+    # Simulate processing that might fail
+    if item == 3:
+        raise ValueError(f"Invalid item: {item}")
     return item * 2
 
 items = [1, 2, 3, 4, 5]
-results = multi_thread(process_item, items, workers=3)
-print(results)  # [2, 4, 6, 8, 10]
+
+# Default behavior: raise on first error with rich traceback
+try:
+    results = multi_thread(process_item, items, workers=3)
+except SystemExit:
+    print("Error occurred and was displayed with rich formatting")
+
+# Continue processing on errors, return None for failed items
+results = multi_thread(process_item, items, workers=3, error_handler='ignore')
+print(results)  # [2, 4, None, 8, 10]
+
+# Log errors to files and continue processing
+results = multi_thread(process_item, items, workers=3, error_handler='log', max_error_files=10)
+print(results)  # [2, 4, None, 8, 10] - errors logged to .cache/speedy_utils/error_logs/
+```
+
+#### Multi-processing with Error Handling
+
+Process items across multiple processes with the same enhanced error handling capabilities.
+
+```python
+from speedy_utils import multi_process
+
+def risky_computation(x):
+    """Computation that might fail for certain inputs."""
+    if x % 5 == 0:
+        raise RuntimeError(f"Cannot process multiples of 5: {x}")
+    return x ** 2
+
+data = list(range(12))
+
+# Process with error logging (continues on errors)
+results = multi_process(
+    risky_computation, 
+    data, 
+    backend='mp',
+    error_handler='log',
+    max_error_files=5
+)
+print(results)  # [0, 1, 4, 9, 16, None, 36, 49, 64, 81, None, 121]
+```
+
+### Enhanced Error Handling
+
+**Speedy Utils** now provides comprehensive error handling for parallel processing with rich formatting and detailed diagnostics.
+
+#### Rich Error Tracebacks
+
+When errors occur, you'll see beautifully formatted tracebacks with:
+
+- **Code context**: Lines of code around the error location
+- **Caller information**: Shows where the parallel function was invoked
+- **Filtered frames**: Focuses on user code, hiding infrastructure details
+- **Color coding**: Easy-to-read formatting with syntax highlighting
+
+#### Error Handling Modes
+
+Choose how to handle errors in parallel processing:
+
+- **`'raise'` (default)**: Stop on first error with detailed traceback
+- **`'ignore'`**: Continue processing, return `None` for failed items  
+- **`'log'`**: Log errors to files and continue processing
+
+#### Error Logging
+
+When using `error_handler='log'`, errors are automatically saved to timestamped files in `.cache/speedy_utils/error_logs/` with full context and stack traces.
+
+#### Progress Reporting with Error Statistics
+
+Progress bars now show real-time error and success counts:
+
+```bash
+Multi-thread [8/10] [00:02<00:00, 3.45it/s, success=8, errors=2, pending=0]
+```
+
+This makes it easy to monitor processing health at a glance.
+
+#### Example: Robust Data Processing
+
+```python
+from speedy_utils import multi_thread
+
+def process_data_record(record):
+    """Process a data record that might have issues."""
+    try:
+        # Your processing logic here
+        value = record['value'] / record['divisor']
+        return {'result': value, 'status': 'success'}
+    except KeyError as e:
+        raise ValueError(f"Missing required field in record: {e}")
+    except ZeroDivisionError:
+        raise ValueError("Division by zero in record")
+
+# Sample data with some problematic records
+data = [
+    {'value': 10, 'divisor': 2},     # OK
+    {'value': 15, 'divisor': 0},     # Will error
+    {'value': 20, 'divisor': 4},     # OK
+    {'value': 25},                   # Missing divisor - will error
+]
+
+# Process with error logging - continues despite errors
+results = multi_thread(
+    process_data_record, 
+    data, 
+    workers=4,
+    error_handler='log',
+    max_error_files=10
+)
+
+print("Results:", results)
+# Output: Results: [{'result': 5.0, 'status': 'success'}, None, {'result': 5.0, 'status': 'success'}, None]
+# Errors are logged to files for later analysis
 ```
 
 ### File I/O

{speedy_utils-1.1.43 → speedy_utils-1.1.45}/README.md

@@ -6,13 +6,49 @@
 
 **Speedy Utils** is a Python utility library designed to streamline common programming tasks such as caching, parallel processing, file I/O, and data manipulation. It provides a collection of decorators, functions, and classes to enhance productivity and performance in your Python projects.
 
+## 🚀 Recent Updates (January 27, 2026)
+
+**Enhanced Error Handling in Parallel Processing:**
+
+- Rich-formatted error tracebacks with code context and syntax highlighting
+- Three error handling modes: 'raise', 'ignore', and 'log'
+- Filtered tracebacks focusing on user code (hiding infrastructure)
+- Real-time progress reporting with error/success statistics
+- Automatic error logging to timestamped files
+- Caller frame information showing where parallel functions were invoked
+
+## Quick Start
+
+### Parallel Processing with Error Handling
+
+```python
+from speedy_utils import multi_thread, multi_process
+
+# Simple parallel processing
+results = multi_thread(lambda x: x * 2, [1, 2, 3, 4, 5])
+# Results: [2, 4, 6, 8, 10]
+
+# Robust processing with error handling
+def process_item(item):
+    if item == 3:
+        raise ValueError(f"Cannot process item {item}")
+    return item * 2
+
+# Continue processing despite errors
+results = multi_thread(process_item, [1, 2, 3, 4, 5], error_handler='log')
+# Results: [2, 4, None, 8, 10] - errors logged automatically
+```
+
 ## Table of Contents
 
+- [🚀 Recent Updates](#-recent-updates-january-27-2026)
+- [Quick Start](#quick-start)
 - [Features](#features)
 - [Installation](#installation)
 - [Usage](#usage)
-  - [Caching](#caching)
   - [Parallel Processing](#parallel-processing)
+  - [Enhanced Error Handling](#enhanced-error-handling)
+  - [Caching](#caching)
   - [File I/O](#file-io)
   - [Data Manipulation](#data-manipulation)
   - [Utility Functions](#utility-functions)
@@ -21,11 +57,12 @@
 ## Features
 
 - **Caching Mechanisms**: Disk-based and in-memory caching to optimize function calls.
-- **Parallel Processing**: Multi-threading, multi-processing, and asynchronous multi-threading utilities.
+- **Parallel Processing**: Multi-threading, multi-processing, and asynchronous multi-threading utilities with enhanced error handling.
 - **File I/O**: Simplified JSON, JSONL, and pickle file handling with support for various file extensions.
 - **Data Manipulation**: Utilities for flattening lists and dictionaries, converting data types, and more.
 - **Timing Utilities**: Tools to measure and log execution time of functions and processes.
 - **Pretty Printing**: Enhanced printing functions for structured data, including HTML tables for Jupyter notebooks.
+- **Enhanced Error Handling**: Rich error tracebacks with code context, configurable error handling modes ('raise', 'ignore', 'log'), and detailed progress reporting.
 
 ## Installation
 
@@ -110,20 +147,133 @@ result = compute_sum(5, 7)  # Retrieved from in-memory cache
 
 ### Parallel Processing
 
-#### Multi-threading
+#### Multi-threading with Enhanced Error Handling
 
-Execute functions concurrently using multiple threads. This approach is straightforward and automatically handles both notebook and Python script executions. In a notebook environment, it delegates the running thread to a separate process. If interrupted, it immediately stops this process, avoiding thread dependency issues where threads continue running until all tasks are completed.
+Execute functions concurrently using multiple threads with comprehensive error handling. The enhanced error handling provides three modes: 'raise' (default), 'ignore', and 'log'. When errors occur, you'll see rich-formatted tracebacks with code context and caller information.
 
 ```python
 from speedy_utils import multi_thread
 
 def process_item(item):
-    # Your processing logic
+    # Simulate processing that might fail
+    if item == 3:
+        raise ValueError(f"Invalid item: {item}")
     return item * 2
 
 items = [1, 2, 3, 4, 5]
-results = multi_thread(process_item, items, workers=3)
-print(results)  # [2, 4, 6, 8, 10]
+
+# Default behavior: raise on first error with rich traceback
+try:
+    results = multi_thread(process_item, items, workers=3)
+except SystemExit:
+    print("Error occurred and was displayed with rich formatting")
+
+# Continue processing on errors, return None for failed items
+results = multi_thread(process_item, items, workers=3, error_handler='ignore')
+print(results)  # [2, 4, None, 8, 10]
+
+# Log errors to files and continue processing
+results = multi_thread(process_item, items, workers=3, error_handler='log', max_error_files=10)
+print(results)  # [2, 4, None, 8, 10] - errors logged to .cache/speedy_utils/error_logs/
+```
+
+#### Multi-processing with Error Handling
+
+Process items across multiple processes with the same enhanced error handling capabilities.
+
+```python
+from speedy_utils import multi_process
+
+def risky_computation(x):
+    """Computation that might fail for certain inputs."""
+    if x % 5 == 0:
+        raise RuntimeError(f"Cannot process multiples of 5: {x}")
+    return x ** 2
+
+data = list(range(12))
+
+# Process with error logging (continues on errors)
+results = multi_process(
+    risky_computation, 
+    data, 
+    backend='mp',
+    error_handler='log',
+    max_error_files=5
+)
+print(results)  # [0, 1, 4, 9, 16, None, 36, 49, 64, 81, None, 121]
+```
+
+### Enhanced Error Handling
+
+**Speedy Utils** now provides comprehensive error handling for parallel processing with rich formatting and detailed diagnostics.
+
+#### Rich Error Tracebacks
+
+When errors occur, you'll see beautifully formatted tracebacks with:
+
+- **Code context**: Lines of code around the error location
+- **Caller information**: Shows where the parallel function was invoked
+- **Filtered frames**: Focuses on user code, hiding infrastructure details
+- **Color coding**: Easy-to-read formatting with syntax highlighting
+
+#### Error Handling Modes
+
+Choose how to handle errors in parallel processing:
+
+- **`'raise'` (default)**: Stop on first error with detailed traceback
+- **`'ignore'`**: Continue processing, return `None` for failed items  
+- **`'log'`**: Log errors to files and continue processing
+
+#### Error Logging
+
+When using `error_handler='log'`, errors are automatically saved to timestamped files in `.cache/speedy_utils/error_logs/` with full context and stack traces.
+
+#### Progress Reporting with Error Statistics
+
+Progress bars now show real-time error and success counts:
+
+```bash
+Multi-thread [8/10] [00:02<00:00, 3.45it/s, success=8, errors=2, pending=0]
+```
+
+This makes it easy to monitor processing health at a glance.
+
+#### Example: Robust Data Processing
+
+```python
+from speedy_utils import multi_thread
+
+def process_data_record(record):
+    """Process a data record that might have issues."""
+    try:
+        # Your processing logic here
+        value = record['value'] / record['divisor']
+        return {'result': value, 'status': 'success'}
+    except KeyError as e:
+        raise ValueError(f"Missing required field in record: {e}")
+    except ZeroDivisionError:
+        raise ValueError("Division by zero in record")
+
+# Sample data with some problematic records
+data = [
+    {'value': 10, 'divisor': 2},     # OK
+    {'value': 15, 'divisor': 0},     # Will error
+    {'value': 20, 'divisor': 4},     # OK
+    {'value': 25},                   # Missing divisor - will error
+]
+
+# Process with error logging - continues despite errors
+results = multi_thread(
+    process_data_record, 
+    data, 
+    workers=4,
+    error_handler='log',
+    max_error_files=10
+)
+
+print("Results:", results)
+# Output: Results: [{'result': 5.0, 'status': 'success'}, None, {'result': 5.0, 'status': 'success'}, None]
+# Errors are logged to files for later analysis
 ```
 
 ### File I/O

{speedy_utils-1.1.43 → speedy_utils-1.1.45}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "speedy-utils"
-version = "1.1.43"
+version = "1.1.45"
 description = "Fast and easy-to-use package for data science"
 authors = [{ name = "AnhVTH", email = "anhvth.226@gmail.com" }]
 readme = "README.md"

speedy_utils-1.1.45/scripts/bug.py

@@ -0,0 +1,12 @@
+import llm_utils
+from speedy_utils import multi_process
+
+llm = llm_utils.LLM(client=8666, timeout=3)
+
+def f(x):
+    return llm('hello world')
+
+
+# multi_process(f, range(10), backend='mp', error_handler='ignore')
+
+f(0)

{speedy_utils-1.1.43 → speedy_utils-1.1.45}/scripts/test_locals.py

@@ -15,5 +15,4 @@ def process_data(item):
 
 if __name__ == '__main__':
     inputs = range(5)
-    ret = multi_process(process_data, inputs, backend='mp', error_handler='log')
-    print(ret)
+    multi_process(process_data, inputs, backend='mp')

{speedy_utils-1.1.43 → speedy_utils-1.1.45}/scripts/test_ray_locals.py

@@ -3,8 +3,7 @@ from speedy_utils import *
 def do_something(x):
     x = 10
     y = 0
-    # Intentionally cause division by zero error for testing
-    _ = x/y
+    x/y
 
 if __name__ == '__main__':
     inputs = range(10)

{speedy_utils-1.1.43 → speedy_utils-1.1.45}/src/llm_utils/lm/llm.py

@@ -16,6 +16,7 @@ from openai.types.chat import ChatCompletionMessageParam
 from pydantic import BaseModel
 
 from speedy_utils.common.utils_io import jdumps
+from speedy_utils import clean_traceback
 
 from .base_prompt_builder import BasePromptBuilder
 from .mixins import (
@@ -159,6 +160,7 @@ class LLM(
         messages.append({'role': 'user', 'content': user_content})
         return cast(Messages, messages)
 
+    @clean_traceback
     def text_completion(
         self, input_data: str | BaseModel | list[dict], **runtime_kwargs
     ) -> list[dict[str, Any]]:
@@ -214,6 +216,7 @@ class LLM(
             results.append(result_dict)
         return results
 
+    @clean_traceback
     def pydantic_parse(
         self,
         input_data: str | BaseModel | list[dict],

{speedy_utils-1.1.43 → speedy_utils-1.1.45}/src/speedy_utils/__init__.py

@@ -58,6 +58,9 @@ from .common.utils_print import (
     fprint,
 )
 
+# Error handling utilities
+from .common.utils_error import clean_traceback, handle_exceptions_with_clean_traceback
+
 # Multi-worker processing
 from .multi_worker.process import multi_process
 from .multi_worker.thread import kill_all_thread, multi_thread
@@ -156,6 +159,9 @@ __all__ = [
     'print_table',
     'setup_logger',
     'log',
+    # Error handling utilities
+    'clean_traceback',
+    'handle_exceptions_with_clean_traceback',
     # Multi-worker processing
     'multi_process',
     'multi_thread',