dlm-murabei-package 1.0.0b0__tar.gz

dlm_murabei_package-1.0.0b0/PKG-INFO

@@ -0,0 +1,142 @@
+Metadata-Version: 2.4
+Name: dlm-murabei-package
+Version: 1.0.0b0
+Summary: 
+Author: Áurea Fonseca
+Author-email: aurea.fonseca@murabei.com
+Requires-Python: >=3.12
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Requires-Dist: numpy (==2.3.2)
+Requires-Dist: pandas (==2.3.1)
+Requires-Dist: pyarrow (==21.0.0)
+Requires-Dist: scipy (==1.16.1)
+Requires-Dist: statsmodels (==0.14.5)
+Requires-Dist: tqdm (==4.67.1)
+Description-Content-Type: text/markdown
+
+# murabei_models — Dynamic Linear Models (DLM)
+Pacote Python para modelos DLM com regressoras exógenas, incluindo ajuste, previsão, atualização online e backtesting, voltado para séries temporais de negócios.
+
+## Instalação
+- Fonte local (desenvolvimento):
+    - git clone <URL_DO_REPO>
+    - cd <PASTA_DO_REPO>
+    - pip install -r src/requirements/python_requirements.txt
+    - bash build.bash
+    - pip install dist/murabei_models-<VERSION>.tar.gz
+
+- Requisitos principais (definidos no pyproject.toml e requirements):
+    - pandas==2.3.1
+    - numpy==2.3.2
+    - statsmodels==0.14.5
+    - scipy==1.16.1
+    - tqdm==4.67.1
+    - pyarrow==21.0.0
+    - dynm (via URL zip: https://github.com/EduardoGPinheiro/dynm/archive/refs/tags/v0.0.2.zip)
+
+Observações:
+- Garanta acesso à Internet para instalar a dependência dynm via URL.
+- Python mínimo recomendado conforme o ambiente do projeto (definir python_requires no empacotamento, se aplicável).
+
+## Exemplo rápido
+Exemplo mínimo baseado em src/dlm/zzz__model_example.py com fit, predict, update e cross-validate:
+
+```python
+import numpy as np
+import pandas as pd
+from dlm.model import DynamicLinearModel
+
+# Dados sintéticos
+np.random.seed(123)
+n = 24
+time = pd.date_range(start="2020-01-01", periods=n, freq="MS")
+x1 = np.linspace(0, 1, n) + 0.1 * np.random.randn(n)
+x2 = np.linspace(1, 0, n) + 0.1 * np.random.randn(n)
+y = 0.7 * x1 - 0.4 * x2 + 0.1 * np.random.randn(n)
+
+data = pd.DataFrame({"time": time, "y": y, "x1": x1, "x2": x2}).set_index("time")
+y = data[["y"]]
+X = data[["x1", "x2"]]
+
+# Instancia e ajusta
+dlm_model = DynamicLinearModel(
+    trend=False,
+    monitoring=True,
+    start_predictions_at=12,
+)
+dlm_model.fit(X=X, y=y)
+
+# Predição
+X_new = np.array([[1.15, 0.05], [1.20, 0.04], [1.25, 0.03]])
+y_pred = dlm_model.predict(X=X_new)
+
+# Atualização online
+y_new = 0.72
+X_new = np.array([[1.18, 0.06]])
+dlm_model.update(y=y_new, X=X_new)
+
+# Backtesting / Cross-validation
+dlm_model.cross_validate(X=X, y=y, K=12)
+
+# Acessos úteis (atributos/métodos)
+_ = dlm_model.get_predictive
+_ = dlm_model.get_params
+_ = dlm_model.get_monitor
+_ = dlm_model.summary
+_ = dlm_model.llk
+```
+
+## Estrutura do projeto
+- src/dlm/
+    - model.py: implementação principal do modelo DLM
+    - data/aux.py: utilitários de dados
+    - update/intervention.py: rotinas de atualização/intervenções
+    - utils/summary.py: sumários e utilidades
+    - zzz__model_example.py: exemplo completo de uso
+
+- src/tests/
+    - test_template.py e data/00__data.parquet para reprodução local
+
+- Empacotamento:
+    - Layout src/, MANIFEST.in e package_data incluindo arquivos .parquet (dlm.data.examples)
+
+- Automação e scripts:
+    - bitbucket-pipelines.yml, build.sh, check_local_linter.bash
+
+- Versionamento:
+    - VERSION (usado no setup.py)
+    - changelog.md
+
+## Desenvolvimento
+- Estilo e lint:
+    - Ruff habilitado com: Pyflakes (F), pycodestyle (E, W), pydocstyle (D, convenção Google), isort (I), pandas-vet (PD), pep8-naming (N), bandit (S)
+    - Largura de linha: 79
+    - ignore: E402, N806, I001
+
+- Fluxo sugerido:
+    - Criar venv e instalar localmente: pip install -e .
+    - Lint: ./check_local_linter.bash
+    - Testes: pytest em src/tests
+
+## Guia de uso
+- Importação principal:
+    - from dlm.model import DynamicLinearModel
+
+- Fluxo típico:
+    - Preparar X (regressoras) e y (alvo) como pandas/numpy
+    - Ajustar: model.fit(X=X, y=y)
+    - Prever: model.predict(X=X_new)
+    - Atualizar online: model.update(y=y_new, X=X_new)
+    - Backtesting: model.cross_validate(X=X, y=y, K=...)
+
+- Atributos e diagnósticos:
+    - model.get_predictive, model.get_params, model.get_monitor, model.summary, model.llk
+
+## Autores
+Áurea Fonseca, Eduardo Pinheiro, Izabel Nolau e Lucas Ribeiro.
+
+## Build local
+

dlm_murabei_package-1.0.0b0/README.md

@@ -0,0 +1,122 @@
+# murabei_models — Dynamic Linear Models (DLM)
+Pacote Python para modelos DLM com regressoras exógenas, incluindo ajuste, previsão, atualização online e backtesting, voltado para séries temporais de negócios.
+
+## Instalação
+- Fonte local (desenvolvimento):
+    - git clone <URL_DO_REPO>
+    - cd <PASTA_DO_REPO>
+    - pip install -r src/requirements/python_requirements.txt
+    - bash build.bash
+    - pip install dist/murabei_models-<VERSION>.tar.gz
+
+- Requisitos principais (definidos no pyproject.toml e requirements):
+    - pandas==2.3.1
+    - numpy==2.3.2
+    - statsmodels==0.14.5
+    - scipy==1.16.1
+    - tqdm==4.67.1
+    - pyarrow==21.0.0
+    - dynm (via URL zip: https://github.com/EduardoGPinheiro/dynm/archive/refs/tags/v0.0.2.zip)
+
+Observações:
+- Garanta acesso à Internet para instalar a dependência dynm via URL.
+- Python mínimo recomendado conforme o ambiente do projeto (definir python_requires no empacotamento, se aplicável).
+
+## Exemplo rápido
+Exemplo mínimo baseado em src/dlm/zzz__model_example.py com fit, predict, update e cross-validate:
+
+```python
+import numpy as np
+import pandas as pd
+from dlm.model import DynamicLinearModel
+
+# Dados sintéticos
+np.random.seed(123)
+n = 24
+time = pd.date_range(start="2020-01-01", periods=n, freq="MS")
+x1 = np.linspace(0, 1, n) + 0.1 * np.random.randn(n)
+x2 = np.linspace(1, 0, n) + 0.1 * np.random.randn(n)
+y = 0.7 * x1 - 0.4 * x2 + 0.1 * np.random.randn(n)
+
+data = pd.DataFrame({"time": time, "y": y, "x1": x1, "x2": x2}).set_index("time")
+y = data[["y"]]
+X = data[["x1", "x2"]]
+
+# Instancia e ajusta
+dlm_model = DynamicLinearModel(
+    trend=False,
+    monitoring=True,
+    start_predictions_at=12,
+)
+dlm_model.fit(X=X, y=y)
+
+# Predição
+X_new = np.array([[1.15, 0.05], [1.20, 0.04], [1.25, 0.03]])
+y_pred = dlm_model.predict(X=X_new)
+
+# Atualização online
+y_new = 0.72
+X_new = np.array([[1.18, 0.06]])
+dlm_model.update(y=y_new, X=X_new)
+
+# Backtesting / Cross-validation
+dlm_model.cross_validate(X=X, y=y, K=12)
+
+# Acessos úteis (atributos/métodos)
+_ = dlm_model.get_predictive
+_ = dlm_model.get_params
+_ = dlm_model.get_monitor
+_ = dlm_model.summary
+_ = dlm_model.llk
+```
+
+## Estrutura do projeto
+- src/dlm/
+    - model.py: implementação principal do modelo DLM
+    - data/aux.py: utilitários de dados
+    - update/intervention.py: rotinas de atualização/intervenções
+    - utils/summary.py: sumários e utilidades
+    - zzz__model_example.py: exemplo completo de uso
+
+- src/tests/
+    - test_template.py e data/00__data.parquet para reprodução local
+
+- Empacotamento:
+    - Layout src/, MANIFEST.in e package_data incluindo arquivos .parquet (dlm.data.examples)
+
+- Automação e scripts:
+    - bitbucket-pipelines.yml, build.sh, check_local_linter.bash
+
+- Versionamento:
+    - VERSION (usado no setup.py)
+    - changelog.md
+
+## Desenvolvimento
+- Estilo e lint:
+    - Ruff habilitado com: Pyflakes (F), pycodestyle (E, W), pydocstyle (D, convenção Google), isort (I), pandas-vet (PD), pep8-naming (N), bandit (S)
+    - Largura de linha: 79
+    - ignore: E402, N806, I001
+
+- Fluxo sugerido:
+    - Criar venv e instalar localmente: pip install -e .
+    - Lint: ./check_local_linter.bash
+    - Testes: pytest em src/tests
+
+## Guia de uso
+- Importação principal:
+    - from dlm.model import DynamicLinearModel
+
+- Fluxo típico:
+    - Preparar X (regressoras) e y (alvo) como pandas/numpy
+    - Ajustar: model.fit(X=X, y=y)
+    - Prever: model.predict(X=X_new)
+    - Atualizar online: model.update(y=y_new, X=X_new)
+    - Backtesting: model.cross_validate(X=X, y=y, K=...)
+
+- Atributos e diagnósticos:
+    - model.get_predictive, model.get_params, model.get_monitor, model.summary, model.llk
+
+## Autores
+Áurea Fonseca, Eduardo Pinheiro, Izabel Nolau e Lucas Ribeiro.
+
+## Build local

dlm_murabei_package-1.0.0b0/pyproject.toml

@@ -0,0 +1,74 @@
+[project]
+name = "dlm-murabei-package"
+version = "1.0.0-b.0"
+description = ""
+authors = [
+    {name = "Áurea Fonseca",email = "aurea.fonseca@murabei.com"},
+    {name = "Eduardo Pinheiro",email = "eduardo.pinheiro@murabei.com"},
+    {name = "Izabel Nolau",email = "izabel.souza@murabei.com"},
+    {name = "Lucas Ribeiro Magalhaes",email = "lucas.magalhaes@murabei.com"}
+]
+
+readme = "README.md"
+requires-python = ">=3.12"
+dependencies = [
+    "pandas==2.3.1", "numpy==2.3.2", "statsmodels==0.14.5",
+    "scipy==1.16.1", "tqdm==4.67.1", "pyarrow==21.0.0"]
+
+# "dynm @ https://github.com/EduardoGPinheiro/dynm/archive/refs/tags/v0.0.2.zip"
+
+
+[tool.poetry]
+packages = [{include = "dlm", from = "src"}]
+
+[build-system]
+requires = ["poetry-core>=2.0.0,<3.0.0"]
+build-backend = "poetry.core.masonry.api"
+
+[tool.ruff]
+line-length = 79 # Quantidade de caracteres que o linter irá considerar na linha
+indent-width = 4
+
+# Pastas onde o linter não atuara. Por padrão ele só lê arquivos python e ignora tudo dentro do .gitignore
+exclude = [
+  "secrets/",
+  "__init__.py",
+  "setup_template.py",
+  "setup.py"
+]
+
+# Lista de linters que serão aplicados. o ruff da suporte a vários e eles podem ser adicionados aqui.
+[tool.ruff.lint]
+select = [
+    "F", # Pyflakes - Procura bugs básicos no codigo
+    "E", # pycodestyle - Pep8
+    "W", # pycodestyle - Pep8
+    "D", # pydocstyle - Docstring
+    "I", # Isort - Imports
+    "PD", # pandas-vet - linter de pandas (não deixa usar inplace nem df como nome de variavel)
+    "N", # pep8-naming - nomes seguindo pep8
+    "S", # flake8-bandit - linter de vulnerabilidade (nmao deixa usar assert)
+]
+
+# Os códigos de erros colocados aqui são ignorados pelo linter. O erro abaixo é em relação ao assert. Está comentado por padrão.
+ignore = [
+  # "S101" # Use of `assert` detected
+  "E402", "N806", "I001"
+]
+
+# ativa um modo plus do ruff
+preview = true
+
+[tool.ruff.format]
+indent-style = "space"
+
+[tool.ruff.lint.pydocstyle]
+convention = "google" # seleciona as docstrings do google como padrão
+
+[tool.ruff.lint.isort]
+no-sections = true
+
+[dependency-groups]
+docs = [
+    "mkdocs (>=1.6.1,<2.0.0)"
+]

dlm_murabei_package-1.0.0b0/src/dlm/data/aux.py

@@ -0,0 +1,287 @@
+"""Auxiliary functions for data manipulation."""
+import numpy as np
+import pandas as pd
+from pandas import DatetimeIndex
+from typing import Union, Tuple, List
+
+
+def custom_mean(y: np.array) -> np.ndarray | float:
+    """Compute the mean of an array with NaN handling.
+
+    This function computes the mean of a NumPy array while ignoring NaN values.
+    If all values are NaN, the result defaults to 0.
+    Works for both 1D and higher-dimensional arrays.
+
+    Args:
+        y (np.ndarray): Input array of numeric values
+        that may contain NaN values.
+
+    Returns:
+        float | np.ndarray:
+            - If `y` is 1D, returns a single float value representing the mean.
+            - If `y` has more than 1 dimension, returns an array with means
+                along axis=0, replacing columns that contain only NaN
+                values with 0.
+
+    Raises:
+        ValueError: If the input is not a NumPy array.
+    """
+    if not isinstance(y, np.ndarray):
+        raise ValueError("Input must be a NumPy array.")
+
+    # Case: 1D array
+    if y.ndim == 1:
+        # If all values are NaN, return 0
+        if np.isnan(y).all():
+            mu0 = 0
+        else:
+            # Compute mean ignoring NaN values
+            mu0 = np.nanmean(y)
+
+    # Case: multi-dimensional array
+    elif y.ndim > 1:
+        # Compute mean along axis=0 while ignoring NaN
+        mu0 = np.nanmean(y, axis=0)
+
+        # Identify columns where all values are NaN
+        all_nan_mask = np.isnan(y).all(axis=0)
+
+        # Replace NaN means with 0 for those columns
+        mu0[all_nan_mask] = 0
+
+    return mu0
+
+
+def custom_std(y: np.ndarray) -> np.ndarray | float:
+    """Compute the standard deviation of an array with NaN handling.
+
+    This function computes the standard deviation of a NumPy array while
+    ignoring NaN values. If all values are NaN, the result defaults to 1.
+    To avoid zero-variance issues, the function enforces a minimum
+    return value of 1e-6.
+
+    Works for both 1D and multi-dimensional arrays.
+
+    Args:
+        y (np.ndarray): Input array of numeric values that may
+            contain NaN values.
+
+    Returns:
+        float | np.ndarray:
+            - If `y` is 1D, returns a single float value representing
+                the standard deviation.
+            - If `y` has more than 1 dimension, returns an array of
+                standard deviations computed along axis=0, replacing
+                columns of all-NaN values with 1.
+
+    Raises:
+        ValueError: If the input is not a NumPy array.
+    """
+    if not isinstance(y, np.ndarray):
+        raise ValueError("Input must be a NumPy array.")
+
+    # Case: 1D array
+    if y.ndim == 1:
+        # If all values are NaN, return 1.0
+        if np.isnan(y).all():
+            s0 = 1.0
+        else:
+            # Compute standard deviation ignoring NaN values
+            s0 = np.nanstd(y)
+
+    # Case: multi-dimensional array
+    elif y.ndim > 1:
+        # Compute std along axis=0 while ignoring NaN values
+        s0 = np.nanstd(y, axis=0)
+
+        # Identify columns where all values are NaN
+        all_nan_mask = np.isnan(y).all(axis=0)
+
+        # Replace std values of all-NaN columns with 1.0
+        s0[all_nan_mask] = 1.0
+
+    # Ensure minimum standard deviation is at least 1e-6
+    s0 = np.maximum(s0, 1e-6)
+
+    return s0
+
+
+def identify_dummies(X: np.ndarray) -> np.ndarray:  # noqa: N803
+    """Identify dummy-variable columns (0/1-only) in a 2D array.
+
+    This function returns a boolean mask indicating which columns of the input
+    array contain only the binary values 0 and 1. It uses vectorized checks
+    with NumPy's isin and a column-wise reduction for efficiency.
+
+    Args:
+        X (np.ndarray): A 2D array where each column is a feature;
+            may have any numeric dtype.
+
+    Returns:
+        np.ndarray: A 1D boolean array of shape (n_features,), where True marks
+            columns that contain only 0s and 1s, and False otherwise.
+
+    Raises:
+        ValueError: If X is not a NumPy array or is not 2-dimensional.
+    """
+    if not isinstance(X, np.ndarray):
+        raise ValueError("X must be a NumPy ndarray.")
+    if X.ndim != 2:
+        raise ValueError("X must be 2-dimensional (n_samples, n_features).")
+
+    # Check membership against {0, 1} for every element,
+    # then reduce along rows to mark columns.
+    # np.isin returns a boolean array; np.all(..., axis=0)
+    # marks columns that are entirely in {0,1}.
+    bool_vector = np.all(np.isin(X, [0, 1]), axis=0)
+
+    return bool_vector
+
+
+def to_time_df(
+    time: Union[np.ndarray, DatetimeIndex, List, None],
+    start: int = 1
+) -> pd.DataFrame:
+    """Convert time-like input into a DataFrame with 'time' and sequential 't'.
+
+    This function converts a time-like input into a DataFrame with two columns:
+    'time' (the provided values as a pandas Series) and 't' (a 1-based
+    sequential index starting from `start`). If `time` is None, returns an
+    empty DataFrame with only column 't' and zero rows.
+
+    Args:
+        time (np.ndarray | DatetimeIndex | list | None): Time values to convert
+            into a DataFrame; accepted inputs include NumPy arrays, pandas
+            DatetimeIndex, Python lists, or None.
+        start (int): Starting value for the 't' sequence; defaults to 1.
+
+    Returns:
+        pd.DataFrame: A DataFrame with columns:
+            - 'time': The input time values as a pandas Series (if provided).
+            - 't': A sequence starting at `start` with step 1, aligned to rows.
+
+    Raises:
+        TypeError: If `time` is not None and not an accepted type
+            (np.ndarray, DatetimeIndex, or list).
+        ValueError: If the provided `time` is not one-dimensional.
+
+    Notes:
+        - When `time` is a DatetimeIndex, it is converted to a pandas Series,
+            preserving datetime dtype semantics.
+        - The 't' column is derived from the DataFrame's RangeIndex plus
+            `start`, producing 1-based-like sequencing without changing the
+            actual index.
+    """
+    # Using an empty Series to construct an empty
+    if time is None:
+        return pd.DataFrame({"t": []})
+
+    # Normalize acceptable inputs to a 1D pandas Series.
+    # - np.ndarray/list: wrapped into Series directly.
+    # - DatetimeIndex: also acceptable; Series preserves datetime nature.
+    if isinstance(time, (np.ndarray, list, DatetimeIndex)):
+        time = pd.Series(time)
+    else:
+        raise TypeError(
+            "`time` must be a np.ndarray, "
+            "pandas.DatetimeIndex, list, or None.")
+
+    # Validate dimensionality: function expects a 1D sequence of time values.
+    if time.ndim != 1:
+        raise ValueError("`time` must be one-dimensional.")
+
+    # Build the output DataFrame with the time values.
+    time_df = pd.DataFrame({"time": time})
+
+    # Create a 1-based sequence starting from `start` without altering
+    # the DataFrame index.
+    time_df["t"] = time_df.index + start
+
+    return time_df
+
+
+def to_numpy(
+    y: Union[np.array, pd.Series, pd.DataFrame, float],
+    X: Union[np.array, pd.DataFrame, None]  # noqa: N803
+) -> Tuple[np.array, Union[np.array, None], pd.DataFrame]:
+    """Convert targets and features to NumPy, returning time metadata.
+
+    Converts `y` to a 1D NumPy array and `X` (if provided) to a NumPy array,
+    preserving row alignment. Also returns a time DataFrame (from index if
+    datetime-like) and inferred frequency when applicable.
+
+    Args:
+        y (np.ndarray | pd.Series | pd.DataFrame | float):
+            Target variable; must be 1D or a single-column DataFrame.
+        X (np.ndarray | pd.DataFrame | None):
+            Feature matrix; if not None, its number of rows must match `y`.
+
+    Returns:
+        Tuple[np.ndarray, np.ndarray | None, pd.DataFrame, str | None]:
+            - y: 1D NumPy array of targets.
+            - X: NumPy array of features or None.
+            - time_df: DataFrame with time info derived from
+                `y.index` when convertible.
+            - freq: Inferred frequency string from datetime index, or None.
+
+    Raises:
+        TypeError: If `y` or `X` are of unsupported types.
+        ValueError: If `y` is not 1D/single-column or if `X` rows mismatch `y`.
+    """
+    # Extract time from y.index when datetime-like; otherwise ignore.
+    if isinstance(y, pd.DataFrame) or isinstance(y, pd.Series):
+        index = y.index
+        if pd.api.types.is_datetime64_any_dtype(index):
+            time = pd.to_datetime(index, errors='coerce')
+        elif pd.api.types.is_numeric_dtype(index):
+            time = None
+        else:
+            try:
+                time_converted = pd.to_datetime(index, errors='coerce')
+                if time_converted.notna().all():
+                    time = time_converted
+                else:
+                    time = None
+            except Exception:
+                print("y.index is not a valid datetime or could not be"
+                      " converted. It will not be used as the time column.")
+    else:
+        time = None
+    time_df = to_time_df(time)
+
+    if time is not None:
+        freq = pd.infer_freq(time)
+    else:
+        freq = None
+
+    # Convert y to 1D ndarray.
+    if isinstance(y, pd.DataFrame):
+        if y.shape[1] == 1:
+            y = y.iloc[:, 0]
+        else:
+            raise ValueError("`y` should be a 1D array, Series,"
+                             " or single-column DataFrame.")
+    if isinstance(y, pd.Series):
+        y = y.to_numpy()
+    elif isinstance(y, np.ndarray):
+        if y.ndim > 1:
+            raise ValueError("`y` must be 1D.")
+    elif isinstance(y, float):
+        y = np.array([y])
+    else:
+        raise TypeError("`y` must be a np.array, Series,"
+                        " or single-column DataFrame.")
+
+    # Convert X to ndarray if provided and validate row count.
+    if X is not None:
+        if isinstance(X, pd.DataFrame):
+            X = X.to_numpy()
+        elif not isinstance(X, np.ndarray):
+            raise TypeError("`X` must be a np.array or pd.DataFrame.")
+
+        if (X.shape[0] > 1):
+            if (X.shape[0] != y.shape[0]) and (X.shape[0] > 1):
+                raise ValueError("`X` and `y` must have the same number"
+                                 " of rows.")
+
+    return y, X, time_df, freq