homebrewlibra 0.1.0__tar.gz

homebrewlibra-0.1.0/LICENSE.txt

@@ -0,0 +1,13 @@
+Copyright (c) 2026, Alexey Glebov
+
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

homebrewlibra-0.1.0/PKG-INFO

@@ -0,0 +1,112 @@
+Metadata-Version: 2.4
+Name: homebrewlibra
+Version: 0.1.0
+Summary: Modular utilities for market data processing and analysis
+License-File: LICENSE.txt
+Author: Alex Glebov
+Author-email: alex.glebov@gmail.com
+Requires-Python: >=3.10
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Provides-Extra: all
+Provides-Extra: chart
+Provides-Extra: data
+Provides-Extra: dev
+Provides-Extra: docs
+Provides-Extra: exchange
+Provides-Extra: io
+Provides-Extra: market
+Provides-Extra: model
+Provides-Extra: plot
+Provides-Extra: test
+Requires-Dist: black (>=24.0) ; extra == "dev"
+Requires-Dist: ccxt (>=4.5) ; extra == "all"
+Requires-Dist: ccxt (>=4.5) ; extra == "exchange"
+Requires-Dist: homebrewlibra[io] ; extra == "data"
+Requires-Dist: homebrewlibra[io] ; extra == "exchange"
+Requires-Dist: homebrewlibra[io] ; extra == "market"
+Requires-Dist: keras (>=3.0) ; extra == "model"
+Requires-Dist: matplotlib (>=3.10) ; extra == "all"
+Requires-Dist: matplotlib (>=3.10) ; extra == "chart"
+Requires-Dist: matplotlib (>=3.10) ; extra == "plot"
+Requires-Dist: mkdocs (>=1.6) ; extra == "docs"
+Requires-Dist: numpy (>=1.24.0)
+Requires-Dist: pandas (>=2.0.0)
+Requires-Dist: pyarrow (>=14.0.0) ; extra == "all"
+Requires-Dist: pyarrow (>=14.0.0) ; extra == "io"
+Requires-Dist: pytest (>=8.0) ; extra == "dev"
+Requires-Dist: pytest-cov (>=5.0) ; extra == "test"
+Requires-Dist: requests (>=2.32) ; extra == "all"
+Requires-Dist: requests (>=2.32) ; extra == "exchange"
+Requires-Dist: ruff (>=0.5) ; extra == "dev"
+Requires-Dist: tensorflow (>=2.20) ; extra == "model"
+Project-URL: Homepage, https://github.com/Alex-Glebov/HomeBrewLibra
+Project-URL: Issues, https://github.com/Alex-Glebov/HomeBrewLibra/issues
+Project-URL: Repository, https://github.com/Alex-Glebov/HomeBrewLibra
+Description-Content-Type: text/markdown
+
+# HomeBrewLibra
+
+Modular utilities for market data processing and analysis.
+
+## Installation
+
+### Minimal (core only)
+```bash
+pip install homebrewlibra
+```
+
+### With I/O helpers (feather, CSV)
+```bash
+pip install "homebrewlibra[io]"
+```
+
+### For data preparation (market + data + I/O)
+```bash
+pip install "homebrewlibra[market,data,io]"
+```
+
+### For exchange connectivity
+```bash
+pip install "homebrewlibra[exchange]"
+```
+
+### With plotting
+```bash
+pip install "homebrewlibra[plot]"
+```
+
+### Everything (backward compatible)
+```bash
+pip install "homebrewlibra[all]"
+```
+
+## Subpackages
+
+| Subpackage | Extras | Dependencies | Description |
+|------------|--------|-------------|-------------|
+| `core` | (none) | pandas, numpy | Shared utilities: timestamp parsing, field mapping |
+| `helper_market` | `[market]` | core | Trade splitting, grouping, peak detection helpers |
+| `helper_data` | `[data]` | core | DataFrame combination, index analysis |
+| `helper_feather` | `[io]` | pyarrow | Feather file I/O |
+| `helper_csv` | `[io]` | (none) | CSV file I/O |
+| `helper_io` | `[io]` | core | Step execution, file caching |
+| `helper_exchange` | `[exchange]` | ccxt, requests | Exchange API connectivity |
+| `helper_plot` | `[plot]` | matplotlib | Plotting tools |
+| `gantt_chart` | `[chart]` | matplotlib | Gantt chart generation |
+
+## Changes in 0.1.0
+
+- **Removed**: `helper_model` (ML utilities). Use `model-core` package instead.
+- **Added**: `core` subpackage to break circular imports.
+- **Added**: Optional extras `[io]`, `[market]`, `[data]`, `[exchange]`, `[plot]`, `[all]`.
+- **Changed**: `pyproject.toml` now declares minimal core deps; heavy deps moved to extras.
+
+## Contributing
+
+Please [fork this project on GitHub](https://github.com/Alex-Glebov/HomeBrewLibra) and send a pull request.
+

homebrewlibra-0.1.0/README.md

@@ -0,0 +1,60 @@
+# HomeBrewLibra
+
+Modular utilities for market data processing and analysis.
+
+## Installation
+
+### Minimal (core only)
+```bash
+pip install homebrewlibra
+```
+
+### With I/O helpers (feather, CSV)
+```bash
+pip install "homebrewlibra[io]"
+```
+
+### For data preparation (market + data + I/O)
+```bash
+pip install "homebrewlibra[market,data,io]"
+```
+
+### For exchange connectivity
+```bash
+pip install "homebrewlibra[exchange]"
+```
+
+### With plotting
+```bash
+pip install "homebrewlibra[plot]"
+```
+
+### Everything (backward compatible)
+```bash
+pip install "homebrewlibra[all]"
+```
+
+## Subpackages
+
+| Subpackage | Extras | Dependencies | Description |
+|------------|--------|-------------|-------------|
+| `core` | (none) | pandas, numpy | Shared utilities: timestamp parsing, field mapping |
+| `helper_market` | `[market]` | core | Trade splitting, grouping, peak detection helpers |
+| `helper_data` | `[data]` | core | DataFrame combination, index analysis |
+| `helper_feather` | `[io]` | pyarrow | Feather file I/O |
+| `helper_csv` | `[io]` | (none) | CSV file I/O |
+| `helper_io` | `[io]` | core | Step execution, file caching |
+| `helper_exchange` | `[exchange]` | ccxt, requests | Exchange API connectivity |
+| `helper_plot` | `[plot]` | matplotlib | Plotting tools |
+| `gantt_chart` | `[chart]` | matplotlib | Gantt chart generation |
+
+## Changes in 0.1.0
+
+- **Removed**: `helper_model` (ML utilities). Use `model-core` package instead.
+- **Added**: `core` subpackage to break circular imports.
+- **Added**: Optional extras `[io]`, `[market]`, `[data]`, `[exchange]`, `[plot]`, `[all]`.
+- **Changed**: `pyproject.toml` now declares minimal core deps; heavy deps moved to extras.
+
+## Contributing
+
+Please [fork this project on GitHub](https://github.com/Alex-Glebov/HomeBrewLibra) and send a pull request.

homebrewlibra-0.1.0/homebrewlibra/core/__init__.py

@@ -0,0 +1,16 @@
+"""Core utilities shared across homebrewlibra submodules.
+
+This module contains low-level helpers with minimal external dependencies
+to avoid circular imports between subpackages.
+"""
+
+from .timestamp import make_col_unique, parse_unix_ts, utc_string_to_timestamp
+from .fields import check_fields, numercal_col
+
+__all__ = [
+    "make_col_unique",
+    "parse_unix_ts",
+    "utc_string_to_timestamp",
+    "check_fields",
+    "numercal_col",
+]

homebrewlibra-0.1.0/homebrewlibra/core/fields.py

@@ -0,0 +1,82 @@
+"""Field name mapping and type coercion utilities."""
+import logging
+from typing import Dict, List, Union
+
+import pandas as pd
+from pandas import DataFrame
+
+logger = logging.getLogger(__name__)
+
+
+def _get_fields_names_dict():
+    """Lazy import to avoid circular deps at module level."""
+    from homebrewlibra.data_tools import __fields_names_dict
+    return __fields_names_dict
+
+
+def check_fields(df: DataFrame, fields_match: Dict[str, str] = None) -> Dict[str, str]:
+    """Auto-detect or validate DataFrame column names against predefined field dictionaries.
+
+    Returns a standardized name-to-column mapping.
+    """
+    logger.setLevel(logging.INFO)
+    if fields_match is None:
+        best_match = 0
+        best_dict = {}
+        fields_names_dict = _get_fields_names_dict()
+        for key, flds in fields_names_dict.items():
+            logger.debug(f"Processing dict {key}")
+            fields_matched = check_fields(df=df, fields_match=flds)
+            if len(fields_matched) == len(flds):
+                logger.debug(f"accepted dict {key}")
+                break
+            else:
+                if len(fields_matched) >= best_match:
+                    best_dict = fields_matched
+                    best_match = len(best_dict)
+                logger.debug(f"best match dict {key}")
+        return fields_matched if best_match == 0 else best_dict
+    else:
+        sv = ""
+        fields_matched = {}
+        for k, v in fields_match.items():
+            vs = str(v)
+            ks = str(k)
+            if ks.startswith("*"):
+                if sv != "" and any(df[sv] == v):
+                    logger.debug(f" value '{v}' for key '{ks}' value found in '{sv}'")
+                    fields_matched[ks] = sv
+                else:
+                    logger.warning(f" value '{v}' for key '{k}' does not exist in field '{sv}'")
+                    continue
+            else:
+                if ks in df.columns:
+                    logger.debug(f" The key '{ks}' already exist as field name ")
+                    fields_matched[ks] = ks
+                    sv = ks
+                elif vs in df.columns:
+                    logger.debug(f" The value '{vs}' for key '{ks}' exist as field ")
+                    fields_matched[ks] = vs
+                    sv = vs
+                else:
+                    logger.warning(f" The value '{vs}' for key '{ks}' does not exist as field ")
+                    continue
+        return fields_matched
+
+
+def numercal_col(df: DataFrame, col: Union[str, List[str]] = None, inplace: bool = True) -> DataFrame:
+    """Convert specified DataFrame columns to numeric via pd.to_numeric."""
+    newdf = df if inplace else df.copy(deep=True)
+    if col is None:
+        return newdf
+    columns = []
+    if isinstance(col, str):
+        col = [col]
+    for icol in col:
+        if icol in df.columns:
+            columns.append(icol)
+    if len(columns) > 0:
+        newdf[columns] = newdf[columns].apply(pd.to_numeric, errors="coerce")
+    else:
+        raise ValueError(f"passed 'col={col}', but should be either list of strings or string ")
+    return newdf

homebrewlibra-0.1.0/homebrewlibra/core/timestamp.py

@@ -0,0 +1,33 @@
+"""Timestamp and date utilities."""
+import pandas as pd
+from pandas import DataFrame
+from datetime import datetime, timezone
+
+try:
+    from homebrewlibra.helper_date import __TIMEFRAME_MAX
+except ImportError:
+    __TIMEFRAME_MAX = 1e11  # seconds threshold
+
+
+def make_col_unique(df: DataFrame, unique_col: str = "trade_id", ignore_index: bool = False) -> DataFrame:
+    """Drop duplicate rows based on a given column, keeping the first occurrence."""
+    if unique_col is not None:
+        df.drop_duplicates(unique_col, keep="first", inplace=True, ignore_index=ignore_index)
+    return df
+
+
+def parse_unix_ts(col):
+    """Auto-detect seconds vs milliseconds and convert to datetime."""
+    if pd.api.types.is_datetime64_any_dtype(col):
+        return col
+    col = pd.to_numeric(col, errors="coerce")
+    if col.max() < __TIMEFRAME_MAX:
+        return pd.to_datetime(col, unit="s", utc=True)
+    else:
+        return pd.to_datetime(col, unit="ms", utc=True)
+
+
+def utc_string_to_timestamp(date_string: str, date_format: str) -> datetime:
+    """Convert a UTC date string to a timezone-aware datetime."""
+    dt_object = datetime.strptime(date_string, date_format)
+    return dt_object.replace(tzinfo=timezone.utc)

homebrewlibra-0.1.0/homebrewlibra/data_tools/__init__.py

@@ -0,0 +1,63 @@
+DEFAULT_TRADES_COLUMNS = ['timestamp', 'id',      'price', 'amount',  'side',     'type',         'cost']
+TRADES_2_OHLCV_COLUMNS = [
+  'date','open','high','low','close','volume',
+  'miscellaneous',
+  'volume_sell',
+  'volume_buy',
+  'customers_sell',
+  'customers_buy',
+
+  ]
+# fields  in order as it come from exchange
+CONVERT_TRADES_COLUMNS = {
+  'kraken': {
+    'price':'price',
+    'amount':'volume',
+    'timestamp':'timestamp',
+    'buy/sell': 'side',
+    'market/limit':'type',
+    'miscellaneous':'misc',
+    'trade_id':'id',
+
+    }
+  }
+__OHLCVD_COLUMNS       = ['date','open','high','low','close','volume']
+__fields_names_kraken={
+  'amount':'volume',
+  'side'  :'buy/sell',
+  '*buy'   :'b',
+  '*sell'  :'s',
+  'type'  :'market/limit',
+  '*market':'m',
+  '*limit' :'l',
+  'price' :'price',
+  'id'    :'trade_id',
+  'timestamp':'timestamp',
+  'cost':'misc'
+
+  }
+__fields_names_freqtrade={
+  'amount' :'amount',
+  'side'   :'side',
+  '*buy'    :'buy',
+  '*sell'   :'sell',
+  'type'   :'type',
+  '*market' :'market',
+  '*limit'  :'limit',
+  'price'  :'price',
+  'id'     :'id',
+  'timestamp':'timestamp',
+  'cost':'cost'
+  }
+__fields_names_dict={
+  'kraken': __fields_names_kraken,
+  'freqtrade': __fields_names_freqtrade
+  }
+
+__all__=[
+  '__TIMEFRAME_MAX',
+  '__TRADES_COLUMNS',
+  '__OHLCVD_COLUMNS',
+  'TRADES_2_OHLCV_COLUMNS',
+  '__fields_names_dict',
+  ]

homebrewlibra-0.1.0/homebrewlibra/data_tools/create_window.py

@@ -0,0 +1,184 @@
+'''
+Created on 27 Feb 2026
+
+@author: alex
+'''
+import numpy as np 
+from typing import Tuple, Optional, Union
+import logging 
+__logger = logging.getLogger(__name__)
+
+
+
+
+def filter_valid_pivots(
+    pivots: np.ndarray,
+    n_time: int,
+    window_size: int = 1, # just current value )
+    horizon: int = 1,
+) -> np.ndarray:
+    """
+    Returns only those pivot indices (window END positions) that are valid,
+    i.e. have enough history before them and enough future after them.
+
+    Parameters
+    ----------
+    pivots : np.ndarray
+        Candidate window-end indices (0-based)
+    n_time : int
+        Total number of time steps in the data
+    window_size : int
+        Required length of input window
+    horizon : int
+        How many steps ahead the target is (0 = predict current end, 1 = next step, ...)
+
+    Returns
+    -------
+    valid_pivots : np.ndarray
+        Filtered array of valid pivot indices, sorted if input was sorted
+    """
+    pivots = np.asarray(pivots, dtype=np.int64).ravel()
+
+    # Minimum pivot index needed to have full window
+    min_pivot = window_size - 1
+
+    # Maximum pivot index so that pivot + horizon is still inside data
+    max_pivot = n_time - horizon -1# 
+
+    valid_mask = (pivots >= min_pivot) & (pivots <= max_pivot)
+
+    return pivots[valid_mask]
+
+
+
+def check_params_values(y, denominator, n_time, n_features,  window_size, horizon, axis):
+# ────────────────────────────────────────────────
+#  Prepare inputs
+# ────────────────────────────────────────────────
+  if axis != 0:
+    raise NotImplementedError("Only axis=0 is currently supported.")
+  if denominator is None:
+    denominate_flag=False
+    denominator=np.ones(shape=(n_time))
+  else:
+    denominate_flag=True
+    denominator = np.asarray(denominator).ravel()
+  if len(denominator) != n_time:
+    raise ValueError("denominator length must match x_orig time dimension")
+  if y is not None and y.shape[0] != n_time:
+    raise ValueError("y must have same time dimension as x_orig")
+  if window_size < 1:
+    raise ValueError("window_size >= 1 required")
+  if horizon < 0:
+    raise ValueError("horizon >= 0 required")
+
+  return denominate_flag
+# ────────────────────────────────────────────────
+#  Prepare inputs
+# ────────────────────────────────────────────────
+
+def create_window(
+    x_orig: np.ndarray,
+    y_orig: Optional[np.ndarray],
+    denominator: np.ndarray| None=None,
+    window_size: int=1, # just one current X
+    horizon: int = 0, # one ahead
+    axis: int = 0,
+    pivots: Optional[np.ndarray] = None,
+) -> Tuple[np.ndarray, np.ndarray]:
+    """
+
+    Creates normalized sliding windows for Conv1D / time-series models.
+    
+    For each valid pivot p (window END index):
+        - Takes x_orig[p - window_size + 1 : p + 1]
+        - Divides the ENTIRE window by denominator[p]  (exactly the formula you wanted)
+        - Collects into 3D array (n_samples, window_size, n_features)
+    
+    Parameters
+    ----------
+    x_orig : np.ndarray
+        Features. Shape (n_time, n_features) or (n_time,) for univariate.
+    y_orig : np.ndarray or None
+        Targets. Shape (n_time,) or (n_time, n_targets).
+    denominator : np.ndarray
+        1D array of length n_time to divide by (value at window END).
+    window_size : int
+        Length of each input window (>= 1).
+    horizon : int
+        How many steps ahead the target is (default 1).
+    axis : int
+        Currently only 0 is supported (time axis). You can extend later.
+    pivots : np.ndarray
+        Candidate window-end indices (0-based). Only valid ones are kept.
+        If pivots is None → uses all possible valid positions.
+    
+    Returns
+    -------
+    X_windows : np.ndarray
+        Shape (n_valid_windows, window_size, n_features)
+        → ready for Conv1D / LSTM / Transformer input.
+    y_matched_or_indices : np.ndarray
+        If y_orig is not None  → actual target values at (pivot + horizon)
+        If y_orig is None      → array of target indices (so you can do y_orig[indices] later)
+        Creates normalized sliding windows.
+
+    """
+    # ────────────────────────────────────────────────
+    #  Prepare inputs
+    # ────────────────────────────────────────────────
+    if x_orig.ndim == 1:
+      x_orig = x_orig.reshape(-1, 1)
+    n_time, n_features = x_orig.shape
+    if y_orig is not None:
+      y_orig = np.asarray(y_orig)
+
+    denominate_flag = check_params_values(y_orig, denominator, n_time, n_features, window_size, horizon, axis)
+
+    # ────────────────────────────────────────────────
+    #  Determine pivots
+    # ────────────────────────────────────────────────
+    if pivots is None:
+        # Use every possible valid end position
+        pivots = np.arange(n_time)
+
+    valid_pivots = filter_valid_pivots(
+        pivots=pivots,
+        n_time=n_time,
+        window_size=window_size,
+        horizon=horizon,
+    )
+
+    num_samples = len(valid_pivots)
+
+    if num_samples == 0:
+        # Early return with correct empty shapes
+        X_empty = np.empty((0, window_size, n_features), dtype=x_orig.dtype)
+        if y_orig is not None:
+            y_empty = np.empty((0,) + y_orig.shape[1:], dtype=y_orig.dtype)
+        else:
+            y_empty = np.empty((0,), dtype=np.int64)
+        return X_empty, y_empty
+
+    # ────────────────────────────────────────────────
+    #  Build X windows
+    # ────────────────────────────────────────────────
+    X_windows = np.empty((num_samples, window_size, n_features), dtype=x_orig.dtype)
+    window_a = np.array(range(-window_size+1,+ 1))
+    for k, p in enumerate(valid_pivots):
+        window_data = x_orig[window_a+p] #(0 : window_size)+p ]
+        if denominate_flag:
+          window_data /= denominator[p]
+        X_windows[k] = window_data
+    # ────────────────────────────────────────────────
+    #  Build targets / indices
+    # ────────────────────────────────────────────────
+    target_indices = valid_pivots + horizon
+
+    if y_orig is not None:
+        y_matched = y_orig[target_indices]
+    else:
+        y_matched = target_indices
+
+    return X_windows, y_matched
+