tinyconformal 0.2.0__tar.gz → 0.2.1__tar.gz

{tinyconformal-0.2.0 → tinyconformal-0.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: tinyconformal
-Version: 0.2.0
+Version: 0.2.1
 Summary: A small toolbox for conformal prediction
 Author-email: Lucas Leão <heylucasleao@gmail.com>
 License-Expression: MIT
@@ -18,9 +18,17 @@ Requires-Dist: tinyshift[plot]; extra == "plot"
 Provides-Extra: notebook
 Requires-Dist: nbformat>=5.10.4; extra == "notebook"
 Requires-Dist: ipykernel>6.7.0; extra == "notebook"
+Provides-Extra: series
+Requires-Dist: statsforecast>=1.4.0; extra == "series"
+Requires-Dist: mlforecast>=0.7.0; extra == "series"
+Requires-Dist: utilsforecast>=0.1.0; extra == "series"
+Provides-Extra: all
+Requires-Dist: tinyshift[notebook,plot,series]; extra == "all"
 Provides-Extra: dev
-Requires-Dist: tinyconformal[notebook,plot]; extra == "dev"
+Requires-Dist: tinyshift[all]; extra == "dev"
 Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: ruff>=0.1.0; extra == "dev"
+Requires-Dist: mypy>=1.0.0; extra == "dev"
 Dynamic: license-file
 
 # TinyConformal
@@ -36,6 +44,8 @@ For more information on a previous project related to Out-of-Bag (OOB) solutions
 - Added support for exactness-bound-based calibration through `ExactnessBound` for ICP and CQR workflows.
 - Added `unlabeled_fit` support for conformal classifiers and regressors, enabling calibration without labeled calibration data when an exactness bound is available.
 - Classifiers can now be calibrated from unlabeled data using pseudo-labels derived from model predictions, while regressors can use a pre-estimated exactness bound to build the conformity scores.
+- Added `tinyconformal.series` support with `ConformalDistributionTimeSeriesRegressor` and `ConformalQuantileTimeSeriesRegressor` for multi-step time series interval forecasting with customizable backtesting strides (`step_size`).
+- Added support for Conformalized Quantile Regression (CQR) on multi-step time series using base estimators producing quantile forecasts.
 
 Previously, `calibrate` used `Balanced Accuracy Score`; it can now also be calibrated with `Matthews Correlation Coefficient` or `Bookmaker Informedness Score` for improved reliability. The `evaluate` method also reports `bm` and `mcc`.
 
@@ -85,6 +95,7 @@ TinyConformal is organized into two main submodules:
 
 - `tinyconformal.classifier`: conformal classifiers for binary classification.
 - `tinyconformal.regressor`: conformal regressors and exactness-bound utilities.
+- `tinyconformal.series`: multi-step conformal prediction for time series forecasting.
 
 ### Classifier submodule
 
@@ -105,6 +116,13 @@ from tinyconformal.regressor import ConformalizedQuantileRegressor
 from tinyconformal.regressor import ExactnessBound
 ```
 
+###  Time Series submodule
+Import from `tinyconformal.series`:
+
+```python
+from tinyconformal.series import ConformalDistributionTimeSeriesRegressor
+```
+
 ### Example
 
 Example usage of `BinaryClassConditionalConformalClassifier`:
@@ -134,12 +152,16 @@ For settings where labeled calibration data are unavailable, you can fit the con
 ```python
 from sklearn.ensemble import RandomForestClassifier
 from tinyconformal.classifier import BinaryMarginalConformalClassifier
+from sklearn.model_selection import cross_val_score
 
 learner = RandomForestClassifier(n_estimators=100, oob_score=True)
 learner.fit(X_train, y_train)
 
+score = cross_val_score(rf, X_train, y_train, cv=5, scoring='accuracy', n_jobs=-1)
+beta = round(np.mean(1 - score), 3)
+
 conformal_classifier = BinaryMarginalConformalClassifier(learner)
-conformal_classifier.unlabeled_fit(X_unlabeled)
+conformal_classifier.unlabeled_fit(X_unlabeled, beta)
 
 predictions = conformal_classifier.predict(X_test)
 ```
@@ -173,6 +195,102 @@ results = conformal_classifier.evaluate(X_test, y_test)
 print(results)
 ```
 
+### Time Series Example
+
+For point-forecast models, `ConformalDistributionTimeSeriesRegressor` extracts signed empirical residuals ($R = \hat{y} - y$) across backtesting windows to build horizon-specific prediction intervals for Nixtla-style learners (MLForecast or StatsForecast):
+
+```python
+from lightgbm import LGBMRegressor
+from mlforecast import MLForecast
+from tinyconformal.series import ConformalDistributionTimeSeriesRegressor
+
+# Wrap a base forecaster
+mlf = MLForecast(
+    models=[LGBMRegressor(random_state=42)],
+    freq="D",
+    lags=[1, 7],
+)
+
+conformal_ts = ConformalDistributionTimeSeriesRegressor(
+    learner=mlf,
+    horizon=7,
+    n_windows=5,
+    step_size=7,  # Window displacement stride
+    alpha=0.10,
+)
+
+conformal_ts.fit(df)
+intervals_df = conformal_ts.predict_interval(h=7)
+```
+
+For models outputting lower and upper quantile forecasts, `ConformalQuantileTimeSeriesRegressor` computes CQR nonconformity scores $E = \max(\hat{q}_{\text{low}} - y, y - \hat{q}_{\text{high}})$ to produce calibrated prediction intervals:
+
+```python
+from lightgbm import LGBMRegressor
+from mlforecast import MLForecast
+from tinyconformal.series import ConformalQuantileTimeSeriesRegressor
+
+# Forecaster configured to output quantile columns
+mlf = MLForecast(
+    models=[LGBMRegressor(objective="quantile", alpha=0.05), LGBMRegressor(objective="quantile", alpha=0.95)],
+    freq="D",
+    lags=[1, 7],
+)
+
+conformal_cqr_ts = ConformalQuantileTimeSeriesRegressor(
+    learner=mlf,
+    horizon=7,
+    n_windows=5,
+    step_size=7,
+    quantile_cols=("LGBM-lo-90", "LGBM-hi-90"),
+    alpha=0.10,
+)
+
+conformal_cqr_ts.fit(df)
+intervals_df = conformal_cqr_ts.predict_interval(h=7)
+```
+
+### Time Series Mechanics: Horizon vs. Step Size
+
+When calibrating over time series, nonconformity scores are extracted by performing sequential backtesting across multiple calibration windows. The calibration movement is controlled by two parameters:
+- `horizon` ($H$): The forecast horizon step count generated in each window.
+- `step_size` ($S$): The stride length used to advance the origin between backtesting windows.
+
+Below are three typical backtesting movement patterns assuming a forecast horizon ($H = 4$):
+
+#### Small `step_size` ($S = 1 < H$) — Overlapping Windows
+The calibration origin advances by 1 step at a time. This creates significant overlap between consecutive forecast windows, maximizing sample size ($n$) for short historical series.
+
+```plaintext
+Time Axis:      | t1 | t2 | t3 | t4 | t5 | t6 | t7 | t8 | t9 | t10|
+------------------------------------------------------------------
+Window 1:       [=== Initial Train ===]  [--- H=4 (t5 to t8) ---]
+Window 2:       [==== Train + 1 ====]    [--- H=4 (t6 to t9) ---]   (Shifted S=1)
+Window 3:       [===== Train + 2 =====]    [--- H=4 (t7 to t10) --] (Shifted S=1)
+```
+
+#### Default `step_size` ($S = H = 4$) — Disjoint Windows
+The calibration origin shifts by the full forecast horizon ($S = H$). Each window starts exactly where the previous forecast ended, eliminating overlap and ensuring independence among calibration residuals.
+
+```
+Time Axis:      | t1 | t2 | t3 | t4 | t5 | t6 | t7 | t8 | t9 | t10| t11| t12|
+----------------------------------------------------------------------------
+Window 1:       [=== Initial Train ===]  [--- H=4 (t5 to t8) ---]
+Window 2:       [======= Expanded Train =======] [--- H=4 (t9 to t12) --] (Shifted S=4)
+```
+
+#### Large step_size ($S = 6 > H$) — Windows with Gaps
+The stride between windows exceeds the forecast horizon ($S > H$). This introduces temporal gaps between evaluation windows, mimicking real-world systems with infrequent retraining schedules.
+```plaintext
+Time Axis:      | t1 | t2 | t3 | t4 | t5 | t6 | t7 | t8 | t9 | t10| t11| t12| t13| t14|
+----------------------------------------------------------------------------------------
+Window 1:       [=== Initial Train ===]  [--- H=4 (t5 to t8) ---]
+               |                      |                          |
+               |<- Evaluated Train -->| <-- Gap (t9, t10) -----> | (Shifted S=6)
+               |                      |                          v
+Window 2:       [============ Expanded Train ============] [--- H=4 (t11 to t14) --]
+```
+
 ## Classes
 
 ### BinaryMarginalConformalClassifier
@@ -207,6 +325,22 @@ print(results)
 - Training via OOB calibration: `fit(X, y, oob=True)`
 - Training via unlabeled calibration: `unlabeled_fit(X, tilde_beta=..., beta=...)` using an exactness bound
 
+### ConformalDistributionTimeSeriesRegressor
+`ConformalDistributionTimeSeriesRegressor` is a multi-step time series conformal regressor compatible with Nixtla interface estimators (`MLForecast` or `StatsForecast`).
+
+Training & Residual extraction via backtesting: `fit(df, step_size=...)`
+
+Multi-step interval forecasting: `predict_interval(h=..., alpha=...)`
+
+### ConformalQuantileTimeSeriesRegressor
+`ConformalQuantileTimeSeriesRegressor` is a multi-step time series conformal quantile regressor (CQR) compatible with Nixtla interface estimators that output quantile predictions.
+
+Training & Nonconformity score calculation via backtesting:` fit(df, step_size=...)`
+
+Multi-step interval forecasting: `predict_interval(h=..., alpha=...)`
+
+ExactnessBound
+
 ### ExactnessBound
 
 `ExactnessBound` provides helper methods to estimate the exactness bound used in unlabeled conformal calibration for ICP and CQR workflows.

{tinyconformal-0.2.0 → tinyconformal-0.2.1}/README.md

@@ -11,6 +11,8 @@ For more information on a previous project related to Out-of-Bag (OOB) solutions
 - Added support for exactness-bound-based calibration through `ExactnessBound` for ICP and CQR workflows.
 - Added `unlabeled_fit` support for conformal classifiers and regressors, enabling calibration without labeled calibration data when an exactness bound is available.
 - Classifiers can now be calibrated from unlabeled data using pseudo-labels derived from model predictions, while regressors can use a pre-estimated exactness bound to build the conformity scores.
+- Added `tinyconformal.series` support with `ConformalDistributionTimeSeriesRegressor` and `ConformalQuantileTimeSeriesRegressor` for multi-step time series interval forecasting with customizable backtesting strides (`step_size`).
+- Added support for Conformalized Quantile Regression (CQR) on multi-step time series using base estimators producing quantile forecasts.
 
 Previously, `calibrate` used `Balanced Accuracy Score`; it can now also be calibrated with `Matthews Correlation Coefficient` or `Bookmaker Informedness Score` for improved reliability. The `evaluate` method also reports `bm` and `mcc`.
 
@@ -60,6 +62,7 @@ TinyConformal is organized into two main submodules:
 
 - `tinyconformal.classifier`: conformal classifiers for binary classification.
 - `tinyconformal.regressor`: conformal regressors and exactness-bound utilities.
+- `tinyconformal.series`: multi-step conformal prediction for time series forecasting.
 
 ### Classifier submodule
 
@@ -80,6 +83,13 @@ from tinyconformal.regressor import ConformalizedQuantileRegressor
 from tinyconformal.regressor import ExactnessBound
 ```
 
+###  Time Series submodule
+Import from `tinyconformal.series`:
+
+```python
+from tinyconformal.series import ConformalDistributionTimeSeriesRegressor
+```
+
 ### Example
 
 Example usage of `BinaryClassConditionalConformalClassifier`:
@@ -109,12 +119,16 @@ For settings where labeled calibration data are unavailable, you can fit the con
 ```python
 from sklearn.ensemble import RandomForestClassifier
 from tinyconformal.classifier import BinaryMarginalConformalClassifier
+from sklearn.model_selection import cross_val_score
 
 learner = RandomForestClassifier(n_estimators=100, oob_score=True)
 learner.fit(X_train, y_train)
 
+score = cross_val_score(rf, X_train, y_train, cv=5, scoring='accuracy', n_jobs=-1)
+beta = round(np.mean(1 - score), 3)
+
 conformal_classifier = BinaryMarginalConformalClassifier(learner)
-conformal_classifier.unlabeled_fit(X_unlabeled)
+conformal_classifier.unlabeled_fit(X_unlabeled, beta)
 
 predictions = conformal_classifier.predict(X_test)
 ```
@@ -148,6 +162,102 @@ results = conformal_classifier.evaluate(X_test, y_test)
 print(results)
 ```
 
+### Time Series Example
+
+For point-forecast models, `ConformalDistributionTimeSeriesRegressor` extracts signed empirical residuals ($R = \hat{y} - y$) across backtesting windows to build horizon-specific prediction intervals for Nixtla-style learners (MLForecast or StatsForecast):
+
+```python
+from lightgbm import LGBMRegressor
+from mlforecast import MLForecast
+from tinyconformal.series import ConformalDistributionTimeSeriesRegressor
+
+# Wrap a base forecaster
+mlf = MLForecast(
+    models=[LGBMRegressor(random_state=42)],
+    freq="D",
+    lags=[1, 7],
+)
+
+conformal_ts = ConformalDistributionTimeSeriesRegressor(
+    learner=mlf,
+    horizon=7,
+    n_windows=5,
+    step_size=7,  # Window displacement stride
+    alpha=0.10,
+)
+
+conformal_ts.fit(df)
+intervals_df = conformal_ts.predict_interval(h=7)
+```
+
+For models outputting lower and upper quantile forecasts, `ConformalQuantileTimeSeriesRegressor` computes CQR nonconformity scores $E = \max(\hat{q}_{\text{low}} - y, y - \hat{q}_{\text{high}})$ to produce calibrated prediction intervals:
+
+```python
+from lightgbm import LGBMRegressor
+from mlforecast import MLForecast
+from tinyconformal.series import ConformalQuantileTimeSeriesRegressor
+
+# Forecaster configured to output quantile columns
+mlf = MLForecast(
+    models=[LGBMRegressor(objective="quantile", alpha=0.05), LGBMRegressor(objective="quantile", alpha=0.95)],
+    freq="D",
+    lags=[1, 7],
+)
+
+conformal_cqr_ts = ConformalQuantileTimeSeriesRegressor(
+    learner=mlf,
+    horizon=7,
+    n_windows=5,
+    step_size=7,
+    quantile_cols=("LGBM-lo-90", "LGBM-hi-90"),
+    alpha=0.10,
+)
+
+conformal_cqr_ts.fit(df)
+intervals_df = conformal_cqr_ts.predict_interval(h=7)
+```
+
+### Time Series Mechanics: Horizon vs. Step Size
+
+When calibrating over time series, nonconformity scores are extracted by performing sequential backtesting across multiple calibration windows. The calibration movement is controlled by two parameters:
+- `horizon` ($H$): The forecast horizon step count generated in each window.
+- `step_size` ($S$): The stride length used to advance the origin between backtesting windows.
+
+Below are three typical backtesting movement patterns assuming a forecast horizon ($H = 4$):
+
+#### Small `step_size` ($S = 1 < H$) — Overlapping Windows
+The calibration origin advances by 1 step at a time. This creates significant overlap between consecutive forecast windows, maximizing sample size ($n$) for short historical series.
+
+```plaintext
+Time Axis:      | t1 | t2 | t3 | t4 | t5 | t6 | t7 | t8 | t9 | t10|
+------------------------------------------------------------------
+Window 1:       [=== Initial Train ===]  [--- H=4 (t5 to t8) ---]
+Window 2:       [==== Train + 1 ====]    [--- H=4 (t6 to t9) ---]   (Shifted S=1)
+Window 3:       [===== Train + 2 =====]    [--- H=4 (t7 to t10) --] (Shifted S=1)
+```
+
+#### Default `step_size` ($S = H = 4$) — Disjoint Windows
+The calibration origin shifts by the full forecast horizon ($S = H$). Each window starts exactly where the previous forecast ended, eliminating overlap and ensuring independence among calibration residuals.
+
+```
+Time Axis:      | t1 | t2 | t3 | t4 | t5 | t6 | t7 | t8 | t9 | t10| t11| t12|
+----------------------------------------------------------------------------
+Window 1:       [=== Initial Train ===]  [--- H=4 (t5 to t8) ---]
+Window 2:       [======= Expanded Train =======] [--- H=4 (t9 to t12) --] (Shifted S=4)
+```
+
+#### Large step_size ($S = 6 > H$) — Windows with Gaps
+The stride between windows exceeds the forecast horizon ($S > H$). This introduces temporal gaps between evaluation windows, mimicking real-world systems with infrequent retraining schedules.
+```plaintext
+Time Axis:      | t1 | t2 | t3 | t4 | t5 | t6 | t7 | t8 | t9 | t10| t11| t12| t13| t14|
+----------------------------------------------------------------------------------------
+Window 1:       [=== Initial Train ===]  [--- H=4 (t5 to t8) ---]
+               |                      |                          |
+               |<- Evaluated Train -->| <-- Gap (t9, t10) -----> | (Shifted S=6)
+               |                      |                          v
+Window 2:       [============ Expanded Train ============] [--- H=4 (t11 to t14) --]
+```
+
 ## Classes
 
 ### BinaryMarginalConformalClassifier
@@ -182,6 +292,22 @@ print(results)
 - Training via OOB calibration: `fit(X, y, oob=True)`
 - Training via unlabeled calibration: `unlabeled_fit(X, tilde_beta=..., beta=...)` using an exactness bound
 
+### ConformalDistributionTimeSeriesRegressor
+`ConformalDistributionTimeSeriesRegressor` is a multi-step time series conformal regressor compatible with Nixtla interface estimators (`MLForecast` or `StatsForecast`).
+
+Training & Residual extraction via backtesting: `fit(df, step_size=...)`
+
+Multi-step interval forecasting: `predict_interval(h=..., alpha=...)`
+
+### ConformalQuantileTimeSeriesRegressor
+`ConformalQuantileTimeSeriesRegressor` is a multi-step time series conformal quantile regressor (CQR) compatible with Nixtla interface estimators that output quantile predictions.
+
+Training & Nonconformity score calculation via backtesting:` fit(df, step_size=...)`
+
+Multi-step interval forecasting: `predict_interval(h=..., alpha=...)`
+
+ExactnessBound
+
 ### ExactnessBound
 
 `ExactnessBound` provides helper methods to estimate the exactness bound used in unlabeled conformal calibration for ICP and CQR workflows.

{tinyconformal-0.2.0 → tinyconformal-0.2.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "tinyconformal"
-version = "0.2.0"
+version = "0.2.1"
 description = "A small toolbox for conformal prediction"
 license = "MIT"
 authors = [{name = "Lucas Leão", email = "heylucasleao@gmail.com"}]
@@ -27,7 +27,19 @@ notebook = [
     "ipykernel>6.7.0",
 ]
 
+series = [
+    "statsforecast>=1.4.0",
+    "mlforecast>=0.7.0",
+    "utilsforecast>=0.1.0",
+]
+
+all = [
+    "tinyshift[series,plot,notebook]",
+]
+
 dev = [
-    "tinyconformal[plot,notebook]",
+    "tinyshift[all]",
     "pytest>=7.0.0",
-]
+    "ruff>=0.1.0",
+    "mypy>=1.0.0",
+]