dfm-python 0.5.3__tar.gz → 0.5.52__tar.gz

{dfm_python-0.5.3 → dfm_python-0.5.52}/MANIFEST.in

@@ -1,12 +1,11 @@
-# Include only the DFM module (src/nowcasting)
+# Include only the DFM module (src/dfm_python)
 # Exclude all application-specific code (adapters, database, scripts, config, etc.)
 
 include README.md
-include LICENSE
 include pyproject.toml
 
-# Include only the nowcasting package
-recursive-include src/nowcasting *.py
+# Include only the dfm_python package
+recursive-include src/dfm_python *.py
 
 # Exclude application-specific directories
 prune adapters
@@ -29,4 +28,5 @@ exclude *.sql
 exclude *.lock
 exclude .gitignore
 exclude .env*
+exclude LICENSE
 

{dfm_python-0.5.3 → dfm_python-0.5.52}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: dfm-python
-Version: 0.5.3
+Version: 0.5.52
 Summary: Dynamic Factor Model (DFM) estimation and nowcasting in Python
 Project-URL: Homepage, https://github.com/yourusername/dfm-python
 Project-URL: Documentation, https://github.com/yourusername/dfm-python#readme
@@ -8,7 +8,6 @@ Project-URL: Repository, https://github.com/yourusername/dfm-python
 Project-URL: Issues, https://github.com/yourusername/dfm-python/issues
 Author: Minkey Chang
 License: MIT
-License-File: LICENSE
 Keywords: dfm,dynamic-factor-model,econometrics,forecasting,nowcasting,time-series
 Classifier: Development Status :: 4 - Beta
 Classifier: Intended Audience :: Science/Research
@@ -330,6 +329,95 @@ result.confidence_interval  # Confidence interval
 result.factors_at_view     # Factor state at view date
 ```
 
+## Mixed-Frequency Data Handling
+
+### Clock Frequency and Target Series
+
+The package supports mixed-frequency data where the **clock frequency** (the frequency at which factors evolve) can differ from the **target series frequency**. The clock must always be the **minimum frequency** among all series (i.e., the fastest frequency).
+
+### Supported Frequency Combinations
+
+| Frequencies | Clock | Target | Status | Description |
+|------------|-------|--------|--------|-------------|
+| `[d]` | `d` | `d` | ✅ | All daily series |
+| `[d, w]` | `d` | `d` | ✅ | Daily clock, weekly series (7-day uniform average) |
+| `[w, m]` | `w` | `m` | ✅ | Weekly clock, monthly series (4-week uniform average) |
+| `[m]` | `m` | `m` | ✅ | All monthly series |
+| `[m, q]` | `m` | `q` | ✅ | Monthly clock, quarterly series (5-month tent kernel) |
+
+### Weekly Clock with Monthly Series
+
+When `clock='w'` and target series is `frequency='m'`:
+
+**Data Structure:**
+- All data is organized at weekly frequency (T × N, where T is weekly time points)
+- Weekly series: Full weekly observations (week-over-week changes)
+- Monthly series: 3-4 NaN values per month, 1 observation per month (week-over-week changes)
+
+**Model Interpretation:**
+- **Weekly data**: Week-over-week changes (전월비 증감)
+- **Monthly data**: 4-week uniform average of weekly changes (주간 전월비 증감의 4주 평균)
+- **DFM factors**: Weekly frequency, week-over-week changes
+- **Monthly observation**: Connected to 4 weekly state variables via uniform weights `[1, 1, 1, 1]`
+
+**Tent Kernel:**
+```python
+('m', 'w'): np.array([1, 1, 1, 1])  # 4-week uniform average
+# Normalized: [0.25, 0.25, 0.25, 0.25]
+```
+
+**Prediction:**
+- `predict(horizon=4)` returns 4 weekly forecasts (clock frequency)
+- Monthly series predictions are also at weekly frequency
+- **To get monthly forecast**: Average the 4 weekly predictions for that month
+- The monthly prediction equals the average of 4 weekly factors (due to uniform average constraint)
+
+**Mathematical Relationship:**
+```
+Monthly observation = c × (f_t + f_{t-1} + f_{t-2} + f_{t-3})
+                     = c × 4 × mean(f_t, f_{t-1}, f_{t-2}, f_{t-3})
+                     = mean(f_t, f_{t-1}, f_{t-2}, f_{t-3})  (when c = 0.25)
+```
+
+Where:
+- `f_t` is the weekly factor at time t
+- `c` is the loading coefficient (constrained to be uniform via R_mat)
+- The constraint `R_mat @ Cc = q` ensures `Cc[0] = Cc[1] = Cc[2] = Cc[3]` (uniform average)
+
+### Monthly Clock with Quarterly Series
+
+When `clock='m'` and target series is `frequency='q'`:
+
+**Tent Kernel:**
+```python
+('q', 'm'): np.array([1, 2, 3, 2, 1])  # 5-month tent kernel
+# Normalized: [0.111, 0.222, 0.333, 0.222, 0.111]
+```
+
+**Model Interpretation:**
+- Quarterly observation = weighted average of 5 monthly factors (tent shape, peaking at middle month)
+- This is the traditional DFM case (quarterly → monthly)
+
+### Prediction Behavior
+
+**Important Notes:**
+1. **Predictions are always at clock frequency**: `predict(horizon=4)` with `clock='w'` returns 4 weekly forecasts
+2. **No NaN in predictions**: All series (including slower-frequency ones) get predictions at clock frequency
+3. **Aggregation for target frequency**: To get target-frequency forecasts, aggregate clock-frequency predictions:
+   - Monthly target with weekly clock: Average 4 weekly predictions per month
+   - Quarterly target with monthly clock: Apply tent kernel `[1, 2, 3, 2, 1]` to 5 monthly predictions
+
+**Example:**
+```python
+# clock='w', target='m', predict(horizon=4)
+X_forecast = model.predict(horizon=4)  # Returns (4 × N) weekly forecasts
+
+# Get monthly forecast for target series
+monthly_idx = 2  # Index of monthly target series
+weekly_forecasts = X_forecast[:, monthly_idx]  # 4 weekly values
+monthly_forecast = np.mean(weekly_forecasts)  # Average to get monthly value
+```
+
 ## Architecture
 
 ### Core Modules
@@ -390,10 +478,16 @@ pytest src/test/test_trainer.py -v
 
 ## Project Status
 
-**Version**: 0.5.1  
+**Version**: 0.5.4  
 **Status**: Stable  
 **Python**: 3.10+  
 
+### What's New in 0.5.4
+
+- **Version consistency**: Unified version numbers across all files
+- **Documentation updates**: Updated README and package documentation
+- **Stability improvements**: Enhanced numerical stability and error handling
+
 ### What's New in 0.5.1
 
 - **New `update()` method**: Replaces legacy `nowcast()` with flexible `update().predict()` pattern

{dfm_python-0.5.3 → dfm_python-0.5.52}/README.md

@@ -274,6 +274,95 @@ result.confidence_interval  # Confidence interval
 result.factors_at_view     # Factor state at view date
 ```
 
+## Mixed-Frequency Data Handling
+
+### Clock Frequency and Target Series
+
+The package supports mixed-frequency data where the **clock frequency** (the frequency at which factors evolve) can differ from the **target series frequency**. The clock must always be the **minimum frequency** among all series (i.e., the fastest frequency).
+
+### Supported Frequency Combinations
+
+| Frequencies | Clock | Target | Status | Description |
+|------------|-------|--------|--------|-------------|
+| `[d]` | `d` | `d` | ✅ | All daily series |
+| `[d, w]` | `d` | `d` | ✅ | Daily clock, weekly series (7-day uniform average) |
+| `[w, m]` | `w` | `m` | ✅ | Weekly clock, monthly series (4-week uniform average) |
+| `[m]` | `m` | `m` | ✅ | All monthly series |
+| `[m, q]` | `m` | `q` | ✅ | Monthly clock, quarterly series (5-month tent kernel) |
+
+### Weekly Clock with Monthly Series
+
+When `clock='w'` and target series is `frequency='m'`:
+
+**Data Structure:**
+- All data is organized at weekly frequency (T × N, where T is weekly time points)
+- Weekly series: Full weekly observations (week-over-week changes)
+- Monthly series: 3-4 NaN values per month, 1 observation per month (week-over-week changes)
+
+**Model Interpretation:**
+- **Weekly data**: Week-over-week changes (전월비 증감)
+- **Monthly data**: 4-week uniform average of weekly changes (주간 전월비 증감의 4주 평균)
+- **DFM factors**: Weekly frequency, week-over-week changes
+- **Monthly observation**: Connected to 4 weekly state variables via uniform weights `[1, 1, 1, 1]`
+
+**Tent Kernel:**
+```python
+('m', 'w'): np.array([1, 1, 1, 1])  # 4-week uniform average
+# Normalized: [0.25, 0.25, 0.25, 0.25]
+```
+
+**Prediction:**
+- `predict(horizon=4)` returns 4 weekly forecasts (clock frequency)
+- Monthly series predictions are also at weekly frequency
+- **To get monthly forecast**: Average the 4 weekly predictions for that month
+- The monthly prediction equals the average of 4 weekly factors (due to uniform average constraint)
+
+**Mathematical Relationship:**
+```
+Monthly observation = c × (f_t + f_{t-1} + f_{t-2} + f_{t-3})
+                     = c × 4 × mean(f_t, f_{t-1}, f_{t-2}, f_{t-3})
+                     = mean(f_t, f_{t-1}, f_{t-2}, f_{t-3})  (when c = 0.25)
+```
+
+Where:
+- `f_t` is the weekly factor at time t
+- `c` is the loading coefficient (constrained to be uniform via R_mat)
+- The constraint `R_mat @ Cc = q` ensures `Cc[0] = Cc[1] = Cc[2] = Cc[3]` (uniform average)
+
+### Monthly Clock with Quarterly Series
+
+When `clock='m'` and target series is `frequency='q'`:
+
+**Tent Kernel:**
+```python
+('q', 'm'): np.array([1, 2, 3, 2, 1])  # 5-month tent kernel
+# Normalized: [0.111, 0.222, 0.333, 0.222, 0.111]
+```
+
+**Model Interpretation:**
+- Quarterly observation = weighted average of 5 monthly factors (tent shape, peaking at middle month)
+- This is the traditional DFM case (quarterly → monthly)
+
+### Prediction Behavior
+
+**Important Notes:**
+1. **Predictions are always at clock frequency**: `predict(horizon=4)` with `clock='w'` returns 4 weekly forecasts
+2. **No NaN in predictions**: All series (including slower-frequency ones) get predictions at clock frequency
+3. **Aggregation for target frequency**: To get target-frequency forecasts, aggregate clock-frequency predictions:
+   - Monthly target with weekly clock: Average 4 weekly predictions per month
+   - Quarterly target with monthly clock: Apply tent kernel `[1, 2, 3, 2, 1]` to 5 monthly predictions
+
+**Example:**
+```python
+# clock='w', target='m', predict(horizon=4)
+X_forecast = model.predict(horizon=4)  # Returns (4 × N) weekly forecasts
+
+# Get monthly forecast for target series
+monthly_idx = 2  # Index of monthly target series
+weekly_forecasts = X_forecast[:, monthly_idx]  # 4 weekly values
+monthly_forecast = np.mean(weekly_forecasts)  # Average to get monthly value
+```
+
 ## Architecture
 
 ### Core Modules
@@ -334,10 +423,16 @@ pytest src/test/test_trainer.py -v
 
 ## Project Status
 
-**Version**: 0.5.1  
+**Version**: 0.5.4  
 **Status**: Stable  
 **Python**: 3.10+  
 
+### What's New in 0.5.4
+
+- **Version consistency**: Unified version numbers across all files
+- **Documentation updates**: Updated README and package documentation
+- **Stability improvements**: Enhanced numerical stability and error handling
+
 ### What's New in 0.5.1
 
 - **New `update()` method**: Replaces legacy `nowcast()` with flexible `update().predict()` pattern

{dfm_python-0.5.3 → dfm_python-0.5.52}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "dfm-python"
-version = "0.5.3"
+version = "0.5.52"
 description = "Dynamic Factor Model (DFM) estimation and nowcasting in Python"
 readme = "README.md"
 requires-python = ">=3.10"
@@ -65,6 +65,11 @@ all = [
 requires = ["hatchling"]
 build-backend = "hatchling.build"
 
+[tool.hatch.metadata]
+# Prevent automatic license-file field addition
+allow-direct-references = false
+allow-license-file = false
+
 [tool.hatch.build.targets.wheel]
 # Include DFM module
 # All application-specific code (adapters, database, scripts, etc.) is automatically excluded
@@ -78,6 +83,11 @@ exclude = [
     "test_venv/**",
 ]
 
+[dependency-groups]
+dev = [
+    "pytest>=9.0.1",
+]
+
 [project.urls]
 Homepage = "https://github.com/yourusername/dfm-python"
 Documentation = "https://github.com/yourusername/dfm-python#readme"

{dfm_python-0.5.3 → dfm_python-0.5.52}/src/dfm_python/__init__.py

@@ -65,7 +65,7 @@ Note: DFMConfig and SeriesConfig are internal implementation details.
 For detailed documentation, see the README.md file and the tutorial notebooks/scripts.
 """
 
-__version__ = "0.5.2"
+__version__ = "0.5.5"
 
 # ============================================================================
 # PUBLIC API DEFINITION
@@ -85,8 +85,8 @@ __version__ = "0.5.2"
 # Configuration (from config/ subpackage)
 from .config import (
     DEFAULT_BLOCK_NAME,
-    ConfigSource, YamlSource, DictSource, HydraSource,
-    MergedConfigSource, make_config_source,
+    ConfigSource, YamlSource, HydraSource,
+    make_config_source,
 )
 # Internal imports (for backward compatibility, but not recommended)
 from .config import DFMConfig, SeriesConfig  # Internal use only
@@ -102,7 +102,9 @@ from .utils.time import calculate_rmse
 # Users can import these directly from dfm_python
 from .lightning import (
     DFMDataModule,
+    DDFMDataModule,  # DDFM-specific DataModule
     DFMDataset,  # Dataset class (usually not needed directly)
+    DDFMDataset,  # DDFM Dataset class (usually not needed directly)
     KalmanFilter,  # Module class
     EMAlgorithm,  # Module class
 )
@@ -111,8 +113,8 @@ from .lightning import (
 from .models.base import BaseFactorModel
 from .models.dfm import DFM
 
-# DDFM high-level API and low-level model (PyTorch is mandatory)
-from .models.ddfm import DDFM, DDFMModel
+# DDFM high-level API (PyTorch is mandatory)
+from .models.ddfm import DDFM
 
 __all__ = [
     # Core classes
@@ -122,8 +124,8 @@ __all__ = [
     # Constants
     'DEFAULT_BLOCK_NAME',
     # Config sources
-    'ConfigSource', 'YamlSource', 'DictSource', 'HydraSource',
-    'MergedConfigSource', 'make_config_source',
+    'ConfigSource', 'YamlSource', 'HydraSource',
+    'make_config_source',
     # Low-level API (functional interface - advanced usage)
     'BaseResult', 'DFMResult', 'DDFMResult', 'calculate_rmse', 'diagnose_series', 'print_series_diagnosis',
 ]
@@ -131,13 +133,14 @@ __all__ = [
 # DDFM high-level API (PyTorch is mandatory)
 __all__.extend([
     'DDFM',  # High-level API class
-    'DDFMModel',  # Low-level implementation (used internally)
 ])
 
 # Lightning modules (mandatory dependency)
 __all__.extend([
     'DFMDataModule',
+    'DDFMDataModule',  # DDFM-specific DataModule
     'DFMDataset',  # Dataset class (usually not needed directly)
+    'DDFMDataset',  # DDFM Dataset class (usually not needed directly)
     'KalmanFilter',  # Module class
     'EMAlgorithm',  # Module class
 ])

{dfm_python-0.5.3 → dfm_python-0.5.52}/src/dfm_python/config/adapter.py

@@ -208,13 +208,16 @@ class YamlSource:
                         except (ValueError, TypeError):
                             release_date = None
                     
+                    # Remove transformation and blocks if present (not in SeriesConfig)
+                    series_data_clean = {k: v for k, v in series_data.items() 
+                                       if k not in ['transformation', 'blocks']}
                     series_list.append(SeriesConfig(
                         series_id=series_id,
-                        series_name=series_data.get('series_name', series_id),
-                        frequency=series_data.get('frequency', 'm'),
-                        transformation=series_data.get('transformation', 'lin'),
-                        blocks=series_data.get('blocks', []),
-                        release_date=release_date
+                        series_name=series_data_clean.get('series_name', series_id),
+                        frequency=series_data_clean.get('frequency', 'm'),
+                        # transformation removed - handled by preprocessing pipeline
+                        # blocks removed - defined in DFMConfig
+                        release_date=series_data_clean.get('release_date', release_date)
                     ))
         
         # If no series loaded from separate files, try to get from main config
@@ -224,17 +227,23 @@ class YamlSource:
                 for series_item in series_data:
                     if isinstance(series_item, str):
                         # Series is just a string ID - create minimal config
-                        series_list.append(SeriesConfig(series_id=series_item, frequency='m', transformation='lin', blocks=[1]))
+                        series_list.append(SeriesConfig(series_id=series_item, frequency='m'))
                     elif isinstance(series_item, dict):
-                        series_list.append(SeriesConfig(**series_item))
+                        # Remove transformation and blocks if present (not in SeriesConfig)
+                        series_item_clean = {k: v for k, v in series_item.items() 
+                                           if k not in ['transformation', 'blocks']}
+                        series_list.append(SeriesConfig(**series_item_clean))
             elif isinstance(series_data, dict):
                 for series_id, series_item in series_data.items():
                     if isinstance(series_item, dict):
                         series_item['series_id'] = series_id
-                        series_list.append(SeriesConfig(**series_item))
+                        # Remove transformation and blocks if present (not in SeriesConfig)
+                        series_item_clean = {k: v for k, v in series_item.items() 
+                                           if k not in ['transformation', 'blocks']}
+                        series_list.append(SeriesConfig(**series_item_clean))
                     elif isinstance(series_item, str):
                         # Series value is just a string - create minimal config
-                        series_list.append(SeriesConfig(series_id=series_id, frequency='m', transformation='lin', blocks=[1]))
+                        series_list.append(SeriesConfig(series_id=series_id, frequency='m'))
         
         # Load block properties from model config (factors, ar_lag, clock)
         # Model config defines all blocks and their properties
@@ -250,28 +259,8 @@ class YamlSource:
         if blocks_dict_raw is not None:
             blocks_properties_from_config.update(blocks_dict_raw)
         
-        # Derive blocks from series configs (new pattern: blocks discovered from series)
-        # Collect all unique block names from series
-        block_names_from_series = set()
-        for series in series_list:
-            if hasattr(series, 'blocks') and series.blocks:
-                # Handle both list of strings and list of ints
-                if isinstance(series.blocks, list):
-                    for block in series.blocks:
-                        if isinstance(block, str):
-                            block_names_from_series.add(block)
-        
-        # If no blocks specified in series, use default single block
-        # Use first block from model config if available, otherwise create default
-        if not block_names_from_series:
-            # Check if model config defines blocks
-            if blocks_properties_from_config:
-                # Use first block from model config
-                first_block_name = list(blocks_properties_from_config.keys())[0]
-                block_names_from_series.add(first_block_name)
-            else:
-                # Create default generic block
-                block_names_from_series.add(DEFAULT_BLOCK_NAME)
+        # Note: SeriesConfig no longer contains blocks information.
+        # Blocks are defined in DFMConfig, not in SeriesConfig.
         
         # Create block dicts for each block
         # Priority: 1) Model config properties, 2) Defaults from main_settings
@@ -296,24 +285,13 @@ class YamlSource:
                     'clock': default_clock
                 }
         
-        # Then, ensure all blocks referenced by series exist
-        for block_name in block_names_from_series:
-            if block_name not in blocks_dict:
-                # Create block with defaults if not in model config
-                blocks_dict[block_name] = {
-                    'factors': 1,
-                    'ar_lag': default_ar_lag,
-                    'clock': default_clock
-                }
-        
-        # Convert block names to list for from_dict() if series use integer block indices
-        # Check if series use integer indices instead of block names
-        block_names_list = None
-        if series_list:
-            first_series_blocks = series_list[0].blocks if hasattr(series_list[0], 'blocks') else []
-            if first_series_blocks and isinstance(first_series_blocks[0], int):
-                # Series use integer indices, need block_names for from_dict()
-                block_names_list = list(blocks_dict.keys())
+        # If no blocks specified in config, use default single block
+        if not blocks_dict:
+            blocks_dict[DEFAULT_BLOCK_NAME] = {
+                'factors': 1,
+                'ar_lag': default_ar_lag,
+                'clock': default_clock
+            }
         
         # Build dictionary for from_dict() which handles type detection
         # Include all main_settings (which may contain DDFM parameters from model config)
@@ -323,10 +301,6 @@ class YamlSource:
             **main_settings
         }
         
-        # Add block_names if series use integer indices
-        if block_names_list is not None:
-            config_dict['block_names'] = block_names_list
-        
         # Use from_dict() which now correctly detects DDFM configs
         # The detection bug has been fixed in schema.py
         return DFMConfig.from_dict(config_dict)
@@ -644,12 +618,16 @@ if HYDRA_AVAILABLE and ConfigStore is not None:
             
             @schema_dataclass
             class SeriesConfigSchema:
-                """Schema for SeriesConfig validation in Hydra."""
+                """Schema for SeriesConfig validation in Hydra.
+                
+                Note: transformation is handled by preprocessing pipeline, not in SeriesConfig.
+                Note: blocks are defined in DFMConfig, not in SeriesConfig.
+                """
                 series_id: str
                 series_name: str
                 frequency: str
-                transformation: str
-                blocks: ListType[int]
+                # transformation removed - handled by preprocessing pipeline
+                # blocks removed - defined in DFMConfig
                 units: Optional[str] = None  # Optional, for display only
             
             @schema_dataclass