dfm-python 0.5.59__tar.gz → 0.6.0__tar.gz

dfm_python-0.6.0/.cursor/agents/test-runner.md

@@ -0,0 +1,118 @@
+---
+name: test-runner
+description: Testing specialist for running pytest, fixing test failures, and verifying implementations. Use proactively after code changes or when user requests testing.
+---
+
+You are a testing specialist focused on running tests, diagnosing failures, and ensuring code correctness.
+
+## Core Responsibilities
+
+When invoked:
+1. **Run tests** using `uv run pytest` (never use system pytest)
+2. **Analyze failures** with clear error messages and stack traces
+3. **Fix issues** directly in the code
+4. **Verify fixes** by re-running tests
+5. **Report results** concisely
+
+## Testing Workflow
+
+### 1. Environment Setup
+- Always use `uv` and virtual environment (`.venv/`)
+- Activate venv: `source .venv/bin/activate` OR use `uv run`
+- Never use system `python` or `pytest` directly
+
+### 2. Running Tests
+
+```bash
+# Run all tests
+uv run pytest
+
+# Run specific test file
+uv run pytest src/test/models/test_dfm.py
+
+# Run with verbose output
+uv run pytest -v
+
+# Run specific test function
+uv run pytest src/test/models/test_dfm.py::TestDFM::test_dfm_fit
+```
+
+### 3. Analyzing Failures
+
+For each failure:
+- **Read the error message** carefully
+- **Check the stack trace** to locate the issue
+- **Identify root cause** (not just symptoms)
+- **Check recent code changes** if applicable
+- **Verify imports and dependencies**
+
+### 4. Fixing Issues
+
+- **Fix directly** in the code (don't just report)
+- **Make minimal changes** to resolve the issue
+- **Preserve existing functionality** unless explicitly changing it
+- **Update tests** if requirements changed (not just to make them pass)
+
+### 5. Verification
+
+- **Re-run the specific failing test** first
+- **Run the full test suite** if multiple tests might be affected
+- **Verify no regressions** introduced
+
+## Common Issues and Fixes
+
+### Import Errors
+- Check if module exists and path is correct
+- Verify `__init__.py` files exist
+- Check if function/class was moved or renamed
+- Update imports to match current code structure
+
+### Assertion Failures
+- Understand what the test expects vs what code produces
+- Check if test expectations are outdated
+- Verify logic correctness, not just make test pass
+
+### Missing Dependencies
+- Install with `uv pip install <package>`
+- Check `pyproject.toml` for required dependencies
+- Verify virtual environment has all packages
+
+### Type Errors
+- Check type hints match actual usage
+- Verify imports for type annotations
+- Update types if code behavior changed
+
+## Output Format
+
+After running tests, provide:
+1. **Summary**: Total tests, passed, failed, skipped
+2. **Failures**: List each failure with:
+   - Test name and location
+   - Error message
+   - Root cause analysis
+   - Fix applied
+3. **Status**: All tests passing or remaining issues
+
+## Best Practices
+
+- **Run tests immediately** when requested
+- **Fix issues proactively** - don't just report them
+- **Use uv and venv** always
+- **Be thorough** but efficient
+- **Verify fixes** before reporting success
+- **Keep test output** concise but informative
+
+## Example Workflow
+
+```
+User: "Run tests for DFM"
+Assistant: 
+1. Runs: `uv run pytest src/test/models/test_dfm.py -v`
+2. Sees failure: ImportError for validate_dfm_initialization
+3. Checks: Function exists in validator.py
+4. Fixes: Updates import path in test file
+5. Re-runs: `uv run pytest src/test/models/test_dfm.py::TestDFM::test_dfm_fit`
+6. Reports: "Fixed import error. All DFM tests passing."
+```
+
+**Remember: Act quickly, fix directly, verify thoroughly.**

dfm_python-0.6.0/=0.11.0

@@ -0,0 +1 @@
+Audited 1 package in 1ms

dfm_python-0.6.0/PKG-INFO

@@ -0,0 +1,56 @@
+Metadata-Version: 2.4
+Name: dfm-python
+Version: 0.6.0
+Summary: Dynamic Factor Model (DFM) estimation and forecasting in Python
+Project-URL: Homepage, https://github.com/yourusername/dfm-python
+Project-URL: Documentation, https://github.com/yourusername/dfm-python#readme
+Project-URL: Repository, https://github.com/yourusername/dfm-python
+Project-URL: Issues, https://github.com/yourusername/dfm-python/issues
+Author: Minkey Chang
+License: MIT
+Keywords: dfm,dynamic-factor-model,econometrics,forecasting,time-series
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering
+Requires-Python: >=3.10
+Requires-Dist: build>=1.4.0
+Requires-Dist: einops>=0.8.1
+Requires-Dist: hydra-core>=1.3.2
+Requires-Dist: matplotlib>=3.5.0
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: pandas>=2.0.0
+Requires-Dist: pyarrow>=22.0.0
+Requires-Dist: pykalman>=0.11.0
+Requires-Dist: pytorch-lightning>=2.0.0
+Requires-Dist: scikit-learn>=1.7.2
+Requires-Dist: scipy>=1.10.0
+Requires-Dist: sktime>=0.40.1
+Requires-Dist: torch>=2.0.0
+Requires-Dist: twine>=6.2.0
+Provides-Extra: all
+Requires-Dist: hydra-core>=1.3.0; extra == 'all'
+Requires-Dist: omegaconf>=2.3.0; extra == 'all'
+Requires-Dist: pytest-cov>=4.0.0; extra == 'all'
+Requires-Dist: pytest>=7.0.0; extra == 'all'
+Requires-Dist: scikit-learn>=1.0.0; extra == 'all'
+Requires-Dist: sktime>=0.27.0; extra == 'all'
+Requires-Dist: sqlalchemy>=2.0.0; extra == 'all'
+Provides-Extra: db
+Requires-Dist: sqlalchemy>=2.0.0; extra == 'db'
+Provides-Extra: deep
+Provides-Extra: dev
+Requires-Dist: pytest-cov>=4.0.0; extra == 'dev'
+Requires-Dist: pytest>=7.0.0; extra == 'dev'
+Requires-Dist: scikit-learn>=1.0.0; extra == 'dev'
+Provides-Extra: forecasting
+Requires-Dist: sktime[forecasting]>=0.40.1; extra == 'forecasting'
+Provides-Extra: hydra
+Requires-Dist: hydra-core>=1.3.0; extra == 'hydra'
+Requires-Dist: omegaconf>=2.3.0; extra == 'hydra'
+Provides-Extra: transform
+Requires-Dist: sktime>=0.27.0; extra == 'transform'

{dfm_python-0.5.59 → dfm_python-0.6.0}/config/ddfm_exchange.yaml

@@ -22,7 +22,7 @@ seed: 3  # Random seed (matching original seed=3)
 
 # Data Handling (used by Dataset, not Model)
 clock: d  # Daily clock frequency (all series are daily)
-target_scaler: null  # No target scaler - data should be pre-scaled before passing to Dataset
+scaler: null  # No scaler - data should be pre-scaled before passing to Dataset
 
 # Frequency mapping (all series are daily)
 # If not specified, all series use clock frequency (d)

{dfm_python-0.5.59 → dfm_python-0.6.0}/config/ddfm_finance.yaml

@@ -20,7 +20,7 @@ seed: null  # Random seed (None uses default)
 
 # Data Handling (used by Dataset, not Model)
 clock: m  # Monthly clock frequency
-target_scaler: standard  # Scaler for target series: 'standard', 'robust', 'minmax', 'maxabs', 'quantile', or null
+scaler: standard  # Scaler for target series: 'standard', 'robust', 'minmax', 'maxabs', 'quantile', or null
 
 # Frequency mapping (optional)
 # If not specified, all series use clock frequency

{dfm_python-0.5.59 → dfm_python-0.6.0}/config/ddfm_macro.yaml

@@ -22,7 +22,7 @@ seed: null  # Random seed (None uses default)
 
 # Data Handling (used by Dataset, not Model)
 clock: w  # Weekly clock frequency (base frequency for latent factors)
-target_scaler: standard  # Scaler for target series: 'standard', 'robust', 'minmax', 'maxabs', 'quantile', or null
+scaler: standard  # Scaler for target series: 'standard', 'robust', 'minmax', 'maxabs', 'quantile', or null
 
 # Frequency mapping for mixed-frequency macro data
 # Clock is weekly (w), so monthly series are slower frequency

dfm_python-0.6.0/config/dfm_exchange.yaml

@@ -0,0 +1,18 @@
+# DFM Configuration for Exchange Rate Datasets
+# Clock: Daily (d) - base frequency for latent factors
+# Data: Daily exchange rate data (all series at same frequency)
+
+# EM Algorithm Parameters
+# NOTE: Exchange tutorial is long daily data (T~7.6k). Full EM can take hours.
+# Keep tutorial runs short by capping iterations and loosening convergence.
+threshold: 1.0e-04  # EM convergence threshold (tutorial-friendly)
+max_iter: 50  # Maximum EM iterations (tutorial-friendly)
+
+# Data Handling
+clock: d  # Daily clock frequency (base frequency for latent factors)
+scaler: standard  # Scaler for all series: 'standard', 'robust', 'minmax', 'maxabs', 'quantile', or null
+# Note: scaler is handled internally by the model, not set on config object
+# Note: nan_method and nan_k are internal constants (not user-configurable)
+num_factors: 2  # Number of factors
+
+

dfm_python-0.6.0/config/dfm_finance.yaml

@@ -0,0 +1,104 @@
+# DFM Configuration for Financial Datasets
+# Based on coded financial indicators (D, E, I, M, P, S, V series)
+# Data: 97 series (D1-D9, E1-E20, I1-I9, M1-M18, P1-P13, S1-S12, V1-V13, plus targets)
+# Target: market_forward_excess_returns
+# Excluded: risk_free_rate, forward_returns
+# Reduced series for testing purposes
+
+# EM Algorithm Parameters
+threshold: 1.0e-05  # EM convergence threshold
+max_iter: 50  # Maximum EM iterations
+
+# Data Handling
+clock: d  # Daily clock frequency (finance.csv uses daily index via date_id)
+scaler: standard  # Scaler for all series: 'standard', 'robust', 'minmax', 'maxabs', 'quantile', or null
+# Note: scaler is handled internally by the model, not set on config object
+# Note: nan_method and nan_k are internal constants (not user-configurable)
+
+# Block Structure for Financial Data (testing configuration)
+# 4 blocks, 2 factors per block (reduced from 3 for faster testing)
+# This still verifies all block mechanisms while reducing state dimension
+blocks:
+  Block_ALL:
+    num_factors: 2  # Reduced from 3 for faster testing
+    series:
+      # Contains all series from Block_EI, Block_MD, and Block_PSV
+      # From Block_EI:
+      - E1
+      - E5
+      - E10
+      - E15
+      - E20
+      - I1
+      - I5
+      - I9
+      # From Block_MD:
+      - M1
+      - M5
+      - M10
+      - M15
+      - M18
+      - D1
+      - D5
+      - D9
+      # From Block_PSV:
+      - P1
+      - P5
+      - P10
+      - P13
+      - S1
+      - S5
+      - S10
+      - V1
+      - V5
+      - V10
+      - market_forward_excess_returns
+
+  Block_EI:
+    num_factors: 2  # Reduced from 3 for faster testing
+    series:
+      # E series: ~6 series
+      - E1
+      - E5
+      - E10
+      - E15
+      - E20
+      # I series: ~4 series
+      - I1
+      - I5
+      - I9
+
+  Block_MD:
+    num_factors: 2  # Reduced from 3 for faster testing
+    series:
+      # M series: ~6 series
+      - M1
+      - M5
+      - M10
+      - M15
+      - M18
+      # D series: ~4 series
+      - D1
+      - D5
+      - D9
+
+  Block_PSV:
+    num_factors: 2  # Reduced from 3 for faster testing
+    series:
+      # P series: ~4 series
+      - P1
+      - P5
+      - P10
+      - P13
+      # S series: ~3 series
+      - S1
+      - S5
+      - S10
+      # V series: ~3 series
+      - V1
+      - V5
+      - V10
+      - market_forward_excess_returns
+
+# Frequency mapping: Not required for single-frequency cases
+# The code will automatically create frequency dict from data columns using clock frequency (d)

dfm_python-0.6.0/config/dfm_macro.yaml

@@ -0,0 +1,84 @@
+# DFM Configuration for Macroeconomic Datasets
+# Based on Korean economic indicators (KO* series)
+# Data: 101 series, mixed weekly/monthly frequency (1985-04-05 to 2025-11-21)
+# Clock: Weekly (w) - base frequency for latent factors
+# Target: KOEQUIPTE (Investment, Equipment, Estimation, SA)
+
+# EM Algorithm Parameters
+threshold: 1.0e-05  # EM convergence threshold
+max_iter: 50  # Maximum EM iterations
+
+# Numerical Stability Parameters
+clip_ar_coefficients: true
+ar_clip_min: -0.99
+ar_clip_max: 0.99
+use_regularization: true
+regularization_scale: 1.0e-05
+
+# Idiosyncratic Component Parameters
+augment_idio: true
+augment_idio_slow: true  # Important for mixed-frequency macro data (weekly clock, monthly indicators)
+idio_rho0: 0.1
+idio_min_var: 1.0e-08
+
+# Data Handling
+clock: w  # Weekly clock frequency (base frequency for latent factors)
+scaler: standard  # Scaler for all series: 'standard', 'robust', 'minmax', 'maxabs', 'quantile', or null
+# Note: scaler is handled internally by the model, not set on config object
+# Note: nan_method and nan_k are internal constants (not user-configurable)
+
+# Tent weights for mixed-frequency aggregation
+# 3-period tent kernel for monthly->weekly aggregation
+tent_weights:
+  m:w: [1, 2, 1]  # 3-period tent kernel for monthly->weekly aggregation
+
+# Block Structure for Macroeconomic Data
+# Single global block for all series
+# Series order: clock-frequency (weekly) first, then slower-frequency (monthly)
+blocks:
+  Block_Global:
+    num_factors: 3  # Number of factors for global block
+    series:
+      # Weekly series (clock frequency) - must come first
+      - KORFBEV  # Retail, Food & Beverages (weekly)
+      - KORWHLS  # Retail, Wholesale (weekly)
+      - KORMCMP  # Retail, Miscellaneous & Comparison (weekly)
+      - KORELEC  # Retail, Electronics (weekly)
+      - KORCNST  # Retail, Construction (weekly)
+      - KORMACH  # Retail, Machinery (weekly)
+      - KOROCOINF  # Retail, Other Consumer & Information (weekly)
+      # Monthly series (slower than clock) - must come after clock-frequency series
+      - KOEMPTOTO  # Total employment (monthly)
+      - KOHWRWEMP  # Hours worked (monthly)
+      - KOWRCDURE  # Durable goods consumption (monthly)
+      - KOCONPRCF  # Production (monthly)
+      - KOEQUIPTE  # Investment, Equipment (monthly, target series)
+      - KOWRCCNSE  # Consumption (monthly, target series)
+      - KOIMPCONA  # Investment, Construction (monthly, target series)
+
+# Frequency mapping for mixed-frequency macro data
+# Clock is weekly (w), so monthly series are slower frequency
+# Grouped format: frequencies are specified as lists grouped by frequency code
+frequency:
+  # Monthly series (slower than weekly clock) - typically end-of-month observations
+  # These are the series used in the tutorial
+  m:
+    - KOEMPTOTO  # Total employment (monthly)
+    - KOHWRWEMP  # Hours worked (monthly)
+    - KOWRCDURE  # Durable goods consumption (monthly)
+    - KOCONPRCF  # Production (monthly)
+    - KOEQUIPTE  # Investment, Equipment (monthly, target series)
+    - KOWRCCNSE  # Consumption (monthly, target series)
+    - KOIMPCONA  # Investment, Construction (monthly, target series)
+  # Weekly series (same as clock frequency - these appear every week like the clock)
+  # This is a mixed-frequency example: clock is weekly, but monthly series are slower
+  # The tutorial now respects YAML frequencies for proper mixed-frequency handling
+  # These weekly series are available in the data at clock frequency
+  w:
+    - KORFBEV  # Retail, Food & Beverages (weekly)
+    - KORWHLS  # Retail, Wholesale (weekly)
+    - KORMCMP  # Retail, Miscellaneous & Comparison (weekly)
+    - KORELEC  # Retail, Electronics (weekly)
+    - KORCNST  # Retail, Construction (weekly)
+    - KORMACH  # Retail, Machinery (weekly)
+    - KOROCOINF  # Retail, Other Consumer & Information (weekly)