aponyx 0.1.18__py3-none-any.whl

aponyx/docs/logging_design.md

@@ -0,0 +1,251 @@
+# Logging Design for Persistence Layer
+
+## Overview
+
+The persistence layer now includes comprehensive logging at INFO and DEBUG levels to provide visibility into data operations without compromising performance or test reliability.
+
+## Design Principles
+
+### 1. **Module-Level Loggers (Recommended Pattern)**
+
+```python
+import logging
+
+logger = logging.getLogger(__name__)
+```
+
+**Why this approach?**
+
+- ✅ **Hierarchical naming**: Logger names follow module structure (`aponyx.persistence.parquet_io`)
+- ✅ **Configurable at any level**: Users can control logging for entire package, submodules, or individual modules
+- ✅ **No global configuration in library code**: The library never calls `logging.basicConfig()` - that's the application's responsibility
+- ✅ **Works with pytest**: Tests run cleanly without logging output unless explicitly enabled
+- ✅ **Standard Python practice**: Follows official Python logging guidelines for library code
+
+**Anti-pattern (what we avoided):**
+```python
+# DON'T DO THIS in library code
+logging.basicConfig(level=logging.INFO)  # Forces configuration on users
+```
+
+### 2. **INFO vs DEBUG Levels**
+
+#### INFO Level (User-Facing Operations)
+Logged at INFO when:
+- Files are saved or loaded
+- Registry operations occur (register, update, remove)
+- Operations complete successfully with summary statistics
+
+**Examples:**
+```python
+logger.info("Saving DataFrame to Parquet: path=%s, rows=%d, columns=%d", path, len(df), len(df.columns))
+logger.info("Registered dataset: name=%s, instrument=%s, rows=%s", name, instrument, row_count)
+```
+
+**Characteristics:**
+- High-level operations
+- Always relevant to users
+- Should appear in production logs
+- Includes key metrics (rows, columns, file sizes)
+
+#### DEBUG Level (Developer Details)
+Logged at DEBUG when:
+- Low-level details about operations
+- File sizes after writing
+- Applied filters and transformations
+- Non-existent directories
+
+**Examples:**
+```python
+logger.debug("Successfully saved %d bytes to %s", path.stat().st_size, path)
+logger.debug("Applied date filter: start=%s, end=%s, resulting_rows=%d", start_date, end_date, len(df))
+```
+
+**Characteristics:**
+- Implementation details
+- Useful for debugging
+- Can be verbose
+- Typically disabled in production
+
+### 3. **Structured Logging with Parameters**
+
+We use **%-formatting with parameters** instead of f-strings in log statements:
+
+```python
+# ✅ CORRECT: Lazy evaluation, structured logging
+logger.info("Loaded %d rows from %s", len(df), path)
+
+# ❌ AVOID: Eager evaluation, string concatenation
+logger.info(f"Loaded {len(df)} rows from {path}")
+```
+
+**Benefits:**
+- **Performance**: String formatting only happens if the log level is enabled
+- **Structured logging**: Log aggregation tools can parse parameters
+- **Consistency**: Standard Python logging best practice
+
+### 4. **Warning Level for Recoverable Errors**
+
+```python
+logger.warning("Failed to extract stats from %s: %s", file_path, str(e))
+```
+
+Used when:
+- An operation fails but execution continues
+- Non-critical errors occur
+- User should be aware but no exception is raised
+
+## What We Log
+
+### Parquet I/O (`parquet_io.py`)
+
+| Operation | Level | Information Logged |
+|-----------|-------|-------------------|
+| `save_parquet()` | INFO | Path, rows, columns, compression |
+| `save_parquet()` | DEBUG | File size after save |
+| `load_parquet()` | INFO | Path, columns filter |
+| `load_parquet()` | INFO | Rows and columns loaded |
+| `load_parquet()` | DEBUG | Date filter details, resulting rows |
+| `list_parquet_files()` | INFO | Number of files found, directory, pattern |
+| `list_parquet_files()` | DEBUG | Directory not exists |
+
+### JSON I/O (`json_io.py`)
+
+| Operation | Level | Information Logged |
+|-----------|-------|-------------------|
+| `save_json()` | INFO | Path, number of top-level keys |
+| `save_json()` | DEBUG | File size after save |
+| `load_json()` | INFO | Path |
+| `load_json()` | DEBUG | Number of top-level keys loaded |
+
+### Registry (`registry.py`)
+
+| Operation | Level | Information Logged |
+|-----------|-------|-------------------|
+| `__init__()` | INFO | Registry path, number of datasets (loaded/created) |
+| `register_dataset()` | INFO | Name, instrument, row count |
+| `register_dataset()` | WARNING | Failed to extract stats from file |
+| `register_dataset()` | DEBUG | Registering non-existent file |
+| `update_dataset_stats()` | INFO | Name, rows, date range after update |
+| `remove_dataset()` | INFO | Name removed, file deleted (if applicable) |
+
+## User Configuration Examples
+
+### Application Setup (Demo/Scripts)
+
+```python
+import logging
+
+# Basic configuration for scripts
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
+    datefmt="%H:%M:%S",
+)
+```
+
+### Fine-Grained Control
+
+```python
+import logging
+
+# Enable DEBUG for parquet_io only
+logging.getLogger("aponyx.persistence.parquet_io").setLevel(logging.DEBUG)
+
+# Disable INFO for json_io
+logging.getLogger("aponyx.persistence.json_io").setLevel(logging.WARNING)
+
+# Enable all persistence layer at INFO
+logging.getLogger("aponyx.persistence").setLevel(logging.INFO)
+```
+
+### Production Configuration
+
+```python
+import logging
+
+# Production: INFO to file, WARNING to console
+file_handler = logging.FileHandler("aponyx.log")
+file_handler.setLevel(logging.INFO)
+
+console_handler = logging.StreamHandler()
+console_handler.setLevel(logging.WARNING)
+
+logging.getLogger("aponyx").addHandler(file_handler)
+logging.getLogger("aponyx").addHandler(console_handler)
+```
+
+### Testing (Silence Logs)
+
+```python
+# In conftest.py or test setup
+import logging
+
+logging.getLogger("aponyx").setLevel(logging.CRITICAL)
+```
+
+Or use pytest's `caplog` fixture:
+```python
+def test_something(caplog):
+    with caplog.at_level(logging.INFO, logger="aponyx.persistence"):
+        # Test code
+        pass
+    
+    assert "Saving DataFrame to Parquet" in caplog.text
+```
+
+## Benefits of This Design
+
+### 1. **Non-Invasive**
+- Library doesn't force logging configuration
+- Users control what they see
+- Tests run silently by default
+
+### 2. **Observable**
+- Users can track data operations
+- Debugging is easier with detailed logs
+- Production monitoring is straightforward
+
+### 3. **Performance**
+- Lazy evaluation (%-formatting)
+- No overhead when logging is disabled
+- Structured data for log aggregation
+
+### 4. **Maintainable**
+- Consistent logging pattern across modules
+- Clear signal-to-noise ratio (INFO vs DEBUG)
+- Easy to add more logging as code evolves
+
+### 5. **Standards-Compliant**
+- Follows Python logging best practices
+- Compatible with enterprise logging infrastructure
+- Works with popular logging frameworks (loguru, structlog)
+
+## Example Output
+
+With `logging.basicConfig(level=logging.INFO)`:
+
+```
+00:08:40 - aponyx.persistence.parquet_io - INFO - Saving DataFrame to Parquet: path=data/cdx_ig_5y.parquet, rows=215, columns=2, compression=snappy
+00:08:41 - aponyx.data.registry - INFO - Loaded existing registry: path=data/.registries/registry.json, datasets=4
+00:08:41 - aponyx.persistence.parquet_io - INFO - Loading Parquet file: path=data/cdx_ig_5y.parquet, columns=all
+00:08:41 - aponyx.persistence.parquet_io - INFO - Loaded 215 rows, 2 columns from data/cdx_ig_5y.parquet
+```
+
+Clean, informative, and actionable.
+
+## Future Enhancements
+
+Potential additions as the project grows:
+
+1. **Metrics Integration**: Add timing decorators for performance monitoring
+2. **Structured Logging**: Migrate to `structlog` for JSON-formatted logs
+3. **Audit Trail**: Add UUID tracking for data lineage
+4. **Performance Logging**: Track I/O performance metrics
+
+---
+
+**Summary**: The logging design follows Python best practices for library code, providing visibility without imposing configuration, and maintaining clean separation between library and application concerns.
+
+**Maintained by:** stabilefrisur  
+**Last Updated:** December 13, 2025

aponyx/docs/performance_evaluation_design.md

@@ -0,0 +1,265 @@
+# Performance Evaluation Design — aponyx
+
+**Status:** ✅ **IMPLEMENTED** (November 9, 2025)
+
+## Objective
+
+The second-stage evaluation feature provides **comprehensive performance analysis of backtest results**. This stage interprets simulation outputs into structured analytical insights. It sits between backtesting and visualization, enabling consistent, extensible, and comparable performance interpretation.
+
+---
+
+## Implementation Status
+
+### ✅ Completed (November 9, 2025)
+
+**Core modules:**
+- ✅ `analyzer.py` - Performance orchestration and evaluation
+- ✅ `decomposition.py` - Return attribution (directional, signal strength, win/loss)
+- ✅ `risk_metrics.py` - Extended metrics (stability, profit factor, tail ratio, consistency)
+- ✅ `registry.py` - JSON-based metadata catalog with CRUD operations
+- ✅ `report.py` - Markdown report generation with comprehensive formatting
+- ✅ `config.py` - PerformanceConfig dataclass with validation
+
+**Key features:**
+- Extended performance metrics beyond basic Sharpe/drawdown
+- Rolling Sharpe analysis with configurable windows
+- Subperiod stability assessment (quarterly by default)
+- Return attribution by direction, signal strength, and win/loss
+- Comprehensive markdown reports
+- Registry-based metadata tracking
+- Integration with backtest layer via BacktestResult objects
+
+**Notebook:**
+- ✅ `05_performance_analysis.ipynb` - Complete workflow notebook (13 cells)
+  - Loads backtest results from Step 4
+  - Reconstructs BacktestResult objects
+  - Runs comprehensive performance analysis
+  - Displays extended metrics tables
+  - Visualizes rolling Sharpe and attribution
+  - Generates reports for all signal-strategy pairs
+  - Registers evaluations in PerformanceRegistry
+
+**Testing:**
+- ✅ Unit tests in `tests/evaluation/performance/`
+- ✅ Registry integration tests
+- ✅ Attribution decomposition tests
+- ✅ Report generation tests
+
+---
+
+## Conceptual Placement
+
+The feature will be implemented under a new subpackage:
+
+```
+src/aponyx/evaluation/performance/
+```
+
+This aligns with the project’s layered architecture, keeping the evaluation domain consistent:
+- `evaluation.suitability` → pre-backtest screening
+- `evaluation.performance` → post-backtest analysis
+
+---
+
+## Core Responsibilities
+
+| Module | Purpose |
+|--------|----------|
+| `analyzer.py` | Orchestrates performance evaluation for one or more backtests |
+| `decomposition.py` | Provides return attribution by signal component, trade side, or regime |
+| `risk_metrics.py` | Computes advanced risk and stability metrics beyond base statistics |
+| `registry.py` | Manages metadata catalog of evaluation runs (JSON-based) |
+| `report.py` | Generates summary reports (Markdown or JSON) |
+| `config.py` | Defines configuration dataclasses and evaluation parameters |
+
+This mirrors the structure of the existing suitability evaluation package for consistency.
+
+---
+
+## Architectural Integration
+
+**Inputs:**
+- One or more `BacktestResult` objects from the backtest layer
+- Optional contextual metadata such as signal name, strategy ID, and market regime labels
+
+**Outputs:**
+- Structured `PerformanceEvaluationResult` data container
+- Optional Markdown or JSON report
+- Registered entry in a `PerformanceRegistry` catalog for traceability
+
+**Dependencies:**
+- Imports allowed from `aponyx.backtest` and `aponyx.persistence`
+- Must not import from `data`, `models`, or `visualization`
+
+---
+
+## Core Components
+
+### Configuration ✅
+A frozen `PerformanceConfig` dataclass defines the evaluation scope including minimum observations, subperiods, risk-free rate, rolling window, and attribution quantiles.
+
+### Evaluation Result Container ✅
+The `PerformanceResult` dataclass stores:
+- Extended metrics (stability, profit factor, tail ratio, rolling Sharpe stats)
+- Attribution results (directional, signal strength, win/loss decomposition)
+- Subperiod stability scores
+- Comprehensive metadata (timestamps, configuration, signal/strategy IDs)
+
+### Analyzer Module ✅
+The `analyze_backtest_performance` function orchestrates:
+- Loading BacktestResult objects
+- Computing extended metrics via risk_metrics module
+- Assessing temporal stability across subperiods
+- Running attribution decomposition
+- Returning structured PerformanceResult
+
+### Registry Pattern ✅
+The `PerformanceRegistry` class provides:
+- JSON-based catalog of evaluation runs
+- CRUD operations: register, get, list (with filters)
+- Metadata tracking (signal, strategy, timestamps, stability scores)
+- Report path management
+
+### Decomposition and Attribution ✅
+The `decomposition.py` module computes:
+- **Directional attribution:** Long vs short P&L contribution
+- **Signal strength attribution:** Quantile-based decomposition (terciles by default)
+- **Win/loss decomposition:** Positive vs negative day breakdown
+
+### Reporting ✅
+The `report.py` module generates:
+- Comprehensive markdown reports with metrics tables
+- Attribution breakdowns with visual formatting
+- Stability analysis and interpretation
+- Timestamped file persistence in `data/workflows/{signal}_{strategy}_{timestamp}/reports/`
+
+---
+
+## Modular Design Principles
+
+| Design Choice | Rationale |
+|----------------|------------|
+| Protocol-based interfaces | Allow integration with third-party analytics libraries such as QuantStats or vectorbt without code rewrites |
+| Functional orchestration with data containers | Simplify testing, serialization, and reproducibility |
+| Registry pattern | Maintain consistent governance across evaluation layers |
+| Separation of computation and interpretation | Enable flexible decision logic and visualization reuse |
+| Optional external adapters | Support gradual integration of external toolkits |
+
+---
+
+## Implementation Status Summary
+
+### ✅ Must-haves (Completed)
+- ✅ Single-signal evaluation of backtest results
+- ✅ Extended performance metrics (stability, profit factor, tail ratio, rolling Sharpe)
+- ✅ Subperiod stability analysis (quarterly by default)
+- ✅ Markdown report generation
+- ✅ Registry-based metadata tracking
+
+### ✅ Should-haves (Completed)
+- ✅ Comparative evaluation across strategies or signals
+- ✅ Basic attribution by trade direction and signal quantile
+- ✅ Rolling performance diagnostics (rolling Sharpe analysis)
+
+### 🔄 Nice-to-haves (Future Work)
+- ⏳ Optional adapters for external analytics libraries (QuantStats, vectorbt)
+- ⏳ Multi-strategy or portfolio-level aggregation
+- ⏳ Advanced attribution by risk source or regime
+- ⏳ Streamlit dashboard integration for interactive review
+
+---
+
+## Directory Layout (Implemented)
+
+```
+src/aponyx/evaluation/
+├── suitability/             # Pre-backtest evaluation
+└── performance/             # Post-backtest analysis ✅
+    ├── __init__.py          # Exports: analyze_backtest_performance, PerformanceRegistry, etc.
+    ├── analyzer.py          # Core orchestration ✅
+    ├── decomposition.py     # Attribution logic ✅
+    ├── risk_metrics.py      # Extended metric computations ✅
+    ├── report.py            # Markdown summaries ✅
+    ├── registry.py          # Metadata catalog ✅
+    └── config.py            # Configuration dataclasses ✅
+
+data/.registries/
+└── performance.json         # Evaluation tracking (runtime) ✅
+```
+
+**Tests:**
+```
+tests/evaluation/performance/
+├── test_analyzer.py         # Core evaluation tests ✅
+├── test_decomposition.py    # Attribution tests ✅
+├── test_risk_metrics.py     # Extended metrics tests ✅
+├── test_report.py           # Report generation tests ✅
+└── test_registry.py         # Registry CRUD tests ✅
+```
+
+---
+
+## Data Flow (Implemented)
+
+```
+BacktestResult(s) from backtest execution
+   ↓
+PerformanceConfig (min_obs=252, n_subperiods=4, rolling_window=63)
+   ↓
+analyze_backtest_performance(backtest_result, config)
+   ├─ compute_extended_metrics() → stability, profit factor, tail ratio
+   ├─ compute_rolling_sharpe() → mean/std of rolling window
+   ├─ compute_attribution() → directional, signal strength, win/loss
+   └─ assess_subperiod_stability() → quarterly stability scores
+   ↓
+PerformanceResult (extended metrics + attribution + metadata)
+   ↓
+generate_performance_report() → Markdown file
+   ↓
+PerformanceRegistry.register_evaluation() → JSON catalog entry
+   ↓
+Visualization Layer (optional)
+```
+
+**Workflow integration:**
+```
+Step 1: Data Download
+Step 2: Signal Computation
+Step 3: Suitability Evaluation
+Step 4: Backtest Execution
+Step 5: Performance Analysis ← NEW
+```
+
+---
+
+## Key Metrics Implemented
+
+### Extended Performance Metrics
+- **Stability Score:** Temporal consistency across subperiods (0-1 scale)
+- **Profit Factor:** Gross wins / gross losses
+- **Tail Ratio:** 95th percentile / 5th percentile return
+- **Rolling Sharpe:** Mean and standard deviation of rolling window Sharpe ratios
+- **Consistency Score:** Percentage of positive subperiods
+- **Recovery Statistics:** Average days to recover from drawdowns, drawdown count
+
+### Attribution Components
+- **Directional:** Long vs short P&L contribution and percentages
+- **Signal Strength:** Tercile-based decomposition (weak/medium/strong)
+- **Win/Loss:** Positive vs negative day breakdown with contribution percentages
+
+---
+
+## Future Extensibility (Unchanged)
+
+- **Adapters:** Optional plugin interfaces for third-party analytics libraries.
+- **Metrics registry:** Dynamically register new performance metrics without changing core logic.
+- **Comparison engine:** Evaluate relative performance across strategies or signals.
+- **Dashboard integration:** Connect with Streamlit for interactive result exploration.
+- **Portfolio extensions:** Aggregate results for multi-strategy analysis.
+
+---
+
+## Design Intent
+
+This design preserves the modular, layered philosophy of the existing system. It introduces a standardized framework for interpreting backtest outputs, leaving room for scalability in analytics, visualization, and external integrations.
+