pytrendy 1.4.0.dev2__tar.gz → 1.4.0.dev4__tar.gz

{pytrendy-1.4.0.dev2 → pytrendy-1.4.0.dev4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pytrendy
-Version: 1.4.0.dev2
+Version: 1.4.0.dev4
 Summary: Trend Detection in Python. Applicable for real-world industry use cases in time series.
 License: MIT License
          
@@ -91,10 +91,13 @@ Most time series tools give you either a "trend component" (via decomposition) o
 
 ## Features
 
-![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Gradual-Cropped.gif)
-![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Abrupt-Cropped.gif)
-![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Noise-Spikes-Cropped.gif)
-![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Noise-Random-Cropped.gif)
+![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Gradual.gif)
+
+![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Abrupt.gif)
+
+![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Noise-Spikes.gif)
+
+![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Noise-Random.gif)
 
 ## Quickstart
 
@@ -115,7 +118,7 @@ results.print_summary()
 Detected: 
 - 3 Uptrends. 
 - 3 Downtrends.
-- 3 Flats.
+- 2 Flats.
 - 0 Noise.
 
 The best detected trend is Down between dates 2025-05-09 - 2025-06-17
@@ -126,13 +129,12 @@ Full Results:
 time_index                                                                               
 1                 Up  2025-01-02  2025-01-24    22     14.013348            5     gradual
 2               Down  2025-01-25  2025-02-05    11    -13.564214            6     gradual
-3               Flat  2025-02-06  2025-02-09     3     -1.168831            9         NaN
-4                 Up  2025-02-10  2025-03-14    32     24.632035            3     gradual
-5               Flat  2025-03-15  2025-03-17     2      5.660173            7         NaN
-6               Down  2025-03-18  2025-04-01    14    -22.721861            4     gradual
-7                 Up  2025-04-02  2025-05-08    36     72.611833            2     gradual
-8               Down  2025-05-09  2025-06-17    39    -73.253968            1     gradual
-9               Flat  2025-06-18  2025-06-30    12      3.910534            8     NaN 
+3               Flat  2025-02-06  2025-02-09     3     -1.168831            8         NaN
+4                 Up  2025-02-10  2025-03-17    35     26.190476            3     gradual
+5               Down  2025-03-18  2025-04-01    14    -22.721861            4     gradual
+6                 Up  2025-04-02  2025-05-08    36     72.611833            2     gradual
+7               Down  2025-05-09  2025-06-17    39    -73.253968            1     gradual
+8               Flat  2025-06-18  2025-06-30    12      3.910534            7         NaN
 -------------------------------------------------------------------------------
 ```
 
@@ -144,7 +146,7 @@ results.filter_segments(direction='Up', sort_by='change_rank')[:3]
 | time_index | direction | start | end | trend_class | change | pct_change | days | total_change | SNR | change_rank |
 |---|---|---|---|---|---|---|---|---|---|---|
 | 7 | Up | 2025-04-02 | 2025-05-08 | gradual | 72.61 | 367.50% | 36 | 72.61 | 21.70 | 2 |
-| 4 | Up | 2025-02-10 | 2025-03-14 | gradual | 24.63 | 169.22% | 32 | 24.63 | 18.87 | 3 |
+| 4 | Up | 2025-02-10 | 2025-03-17 | gradual | 26.19 | 179.93% | 35 | 26.19 | 18.61 | 3 |
 | 1 | Up | 2025-01-02 | 2025-01-24 | gradual | 14.01 | 104.41% | 22 | 14.01 | 22.21 | 5 |
 
 `filter_segments` ranks segments by magnitude (`change_rank`). See the [API reference](https://russellsb.github.io/pytrendy/main/reference/pytrendy/io/results_pytrendy/#pytrendy.io.results_pytrendy.PyTrendyResults.filter_segments) for all filter and sort options.

{pytrendy-1.4.0.dev2 → pytrendy-1.4.0.dev4}/README.md

@@ -28,10 +28,13 @@ Most time series tools give you either a "trend component" (via decomposition) o
 
 ## Features
 
-![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Gradual-Cropped.gif)
-![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Abrupt-Cropped.gif)
-![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Noise-Spikes-Cropped.gif)
-![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Noise-Random-Cropped.gif)
+![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Gradual.gif)
+
+![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Abrupt.gif)
+
+![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Noise-Spikes.gif)
+
+![](https://raw.githubusercontent.com/RussellSB/pytrendy/refs/heads/develop/plots/Noise-Random.gif)
 
 ## Quickstart
 
@@ -52,7 +55,7 @@ results.print_summary()
 Detected: 
 - 3 Uptrends. 
 - 3 Downtrends.
-- 3 Flats.
+- 2 Flats.
 - 0 Noise.
 
 The best detected trend is Down between dates 2025-05-09 - 2025-06-17
@@ -63,13 +66,12 @@ Full Results:
 time_index                                                                               
 1                 Up  2025-01-02  2025-01-24    22     14.013348            5     gradual
 2               Down  2025-01-25  2025-02-05    11    -13.564214            6     gradual
-3               Flat  2025-02-06  2025-02-09     3     -1.168831            9         NaN
-4                 Up  2025-02-10  2025-03-14    32     24.632035            3     gradual
-5               Flat  2025-03-15  2025-03-17     2      5.660173            7         NaN
-6               Down  2025-03-18  2025-04-01    14    -22.721861            4     gradual
-7                 Up  2025-04-02  2025-05-08    36     72.611833            2     gradual
-8               Down  2025-05-09  2025-06-17    39    -73.253968            1     gradual
-9               Flat  2025-06-18  2025-06-30    12      3.910534            8     NaN 
+3               Flat  2025-02-06  2025-02-09     3     -1.168831            8         NaN
+4                 Up  2025-02-10  2025-03-17    35     26.190476            3     gradual
+5               Down  2025-03-18  2025-04-01    14    -22.721861            4     gradual
+6                 Up  2025-04-02  2025-05-08    36     72.611833            2     gradual
+7               Down  2025-05-09  2025-06-17    39    -73.253968            1     gradual
+8               Flat  2025-06-18  2025-06-30    12      3.910534            7         NaN
 -------------------------------------------------------------------------------
 ```
 
@@ -81,7 +83,7 @@ results.filter_segments(direction='Up', sort_by='change_rank')[:3]
 | time_index | direction | start | end | trend_class | change | pct_change | days | total_change | SNR | change_rank |
 |---|---|---|---|---|---|---|---|---|---|---|
 | 7 | Up | 2025-04-02 | 2025-05-08 | gradual | 72.61 | 367.50% | 36 | 72.61 | 21.70 | 2 |
-| 4 | Up | 2025-02-10 | 2025-03-14 | gradual | 24.63 | 169.22% | 32 | 24.63 | 18.87 | 3 |
+| 4 | Up | 2025-02-10 | 2025-03-17 | gradual | 26.19 | 179.93% | 35 | 26.19 | 18.61 | 3 |
 | 1 | Up | 2025-01-02 | 2025-01-24 | gradual | 14.01 | 104.41% | 22 | 14.01 | 22.21 | 5 |
 
 `filter_segments` ranks segments by magnitude (`change_rank`). See the [API reference](https://russellsb.github.io/pytrendy/main/reference/pytrendy/io/results_pytrendy/#pytrendy.io.results_pytrendy.PyTrendyResults.filter_segments) for all filter and sort options.

{pytrendy-1.4.0.dev2 → pytrendy-1.4.0.dev4}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "pytrendy"
-version = "1.4.0.dev2"
+version = "1.4.0.dev4"
 description = "Trend Detection in Python. Applicable for real-world industry use cases in time series."
 authors = [
     { name = "Russell Sammut Bonnici", email = "r.sammutbonnici@gmail.com" },

{pytrendy-1.4.0.dev2 → pytrendy-1.4.0.dev4}/pytrendy/detect_trends.py

@@ -41,6 +41,7 @@ def detect_trends(df: pd.DataFrame, date_col: str, value_col: str, plot=True, me
             Optional parameters to customize detection heuristics. Supported keys:
 
             - **abrupt_padding** (`int`): Number of days to pad around abrupt transitions. Defaults to `0`.
+            - **gradual_padding** (`int`): Number of days to pad after gradual trend ends. Defaults to `0`.
             - **avoid_noise** (`bool`): Whether to avoid noisy segments in trend detection. Defaults to `True`.
         plot_params (dict, optional):
             Optional dict to customise plot appearance. Only used when `plot` is `True`. Supported keys:
@@ -83,6 +84,7 @@ def detect_trends(df: pd.DataFrame, date_col: str, value_col: str, plot=True, me
     # Configures trend detection heuristics
     method_params = {
         'abrupt_padding': method_params.get('abrupt_padding', 0),
+        'gradual_padding': method_params.get('gradual_padding', 0),
         'avoid_noise': method_params.get('avoid_noise', True),
     }
 

{pytrendy-1.4.0.dev2 → pytrendy-1.4.0.dev4}/pytrendy/post_processing/segments_refine/__init__.py

@@ -30,6 +30,7 @@ def refine_segments(df: pd.DataFrame, value_col: str, segments: list[dict], meth
         method_params (dict): Optional parameters for abrupt padding and control. Supported keys:
 
             - **abrupt_padding** (`int`): Number of days to pad. Defaults to `0`.
+            - **gradual_padding** (`int`): Number of days to extend gradual segment ends forward. Defaults to `0`.
 
     Returns:
         list: Final refined segment list.
@@ -40,7 +41,7 @@ def refine_segments(df: pd.DataFrame, value_col: str, segments: list[dict], meth
     segments_refined = classify_trends(df, value_col, segments_refined)
     segments_refined = group_segments(segments_refined) # grouping 1st pass: sporadic flats & noises
 
-    segments_refined = expand_contract_segments(df, value_col, segments_refined) # for gradual
+    segments_refined = expand_contract_segments(df, value_col, segments_refined, method_params) # for gradual + gradual padding
     segments_refined = shave_abrupt_trends(df, value_col, segments_refined, method_params) # for abrupt
 
     segments_refined = clean_artifacts(df, value_col, segments_refined, method_params) # cleans overlaps etc from expand/contract

{pytrendy-1.4.0.dev2 → pytrendy-1.4.0.dev4}/pytrendy/post_processing/segments_refine/abrupt_shaving.py

@@ -134,6 +134,10 @@ def shave_abrupt_trends(df: pd.DataFrame, value_col: str, segments: list[dict],
             if segment['direction'] not in ['Up', 'Down'] or segment['trend_class'] != 'abrupt': 
                 continue
 
+            # Skip segments already padded in first pass; .get returns False when key absent (unpadded segments)
+            if second_pass and segment.get('padded', False):
+                continue
+
             abrupt_start = pd.to_datetime(segment['start'])
             abrupt_end = pd.to_datetime(segment['end'])
 

{pytrendy-1.4.0.dev2 → pytrendy-1.4.0.dev4}/pytrendy/post_processing/segments_refine/artifact_cleanup.py

@@ -259,7 +259,10 @@ def clean_artifacts(df: pd.DataFrame, value_col: str, segments_refined: list[dic
         is_flat = segment['direction'] == 'Flat'
         is_gradual = ('trend_class' in segment and segment['trend_class'] == 'gradual')
         is_abrupt = ('trend_class' in segment and segment['trend_class'] == 'abrupt')
-        is_padded = is_abrupt and ('padded' in segment) and (segment['padded'] == True)
+        # Covers both abrupt_padding and gradual_padding: padded segments must
+        # not be reclassified as Flat by the trend_too_flat / trend_too_small
+        # checks below, since the extension was intentional.
+        is_padded = ('padded' in segment) and (segment['padded'] == True)
         is_small = len(df_segment) <= 5
 
         # Edge case 1: Check SNR for trend but noise
@@ -316,7 +319,9 @@ def clean_artifacts(df: pd.DataFrame, value_col: str, segments_refined: list[dic
             segment['direction'] = 'Noise' 
             if 'trend_class' in segment: del segment['trend_class']
 
-        if trend_ends_too_close or trend_too_small or trend_too_flat:
+        # Padded segments are protected from flat reclassification — the extension
+        # into flat regions is intentional and the original trend still holds.
+        if trend_ends_too_close or trend_too_small or (trend_too_flat and not is_padded):
             segment['direction'] = 'Flat' 
             if 'trend_class' in segment: del segment['trend_class']
         

{pytrendy-1.4.0.dev2 → pytrendy-1.4.0.dev4}/pytrendy/post_processing/segments_refine/gradual_expand_contract.py

@@ -8,17 +8,21 @@ from copy import deepcopy
 from .update_neighbours import update_prev_segment, update_next_segment
 
 
-def expand_contract_segments(df: pd.DataFrame, value_col: str, segments: list[dict]) -> list[dict]:
+def expand_contract_segments(df: pd.DataFrame, value_col: str, segments: list[dict], method_params: dict) -> list[dict]:
     """
     Refines segment boundaries by expanding or contracting based on local extrema.
 
     Examines ±7 days around each segment's start and end to find stronger turning points.
     Skips segments classified as 'abrupt' to preserve their precision.
+    Optionally pads gradual segments into adjacent flat regions when ``gradual_padding > 0``.
 
     Args:
         df (pd.DataFrame): Time series DataFrame.
         value_col (str): Name of the signal column.
         segments (list): List of segment dictionaries.
+        method_params (dict): Detection parameters. Supported keys:
+
+            - **gradual_padding** (`int`): Days to extend gradual segment ends forward. Defaults to `0`.
 
     Returns:
         list: Refined segment list with updated boundaries.
@@ -114,4 +118,62 @@ def expand_contract_segments(df: pd.DataFrame, value_col: str, segments: list[di
             segments_refined[i]['end'] = new_end.strftime('%Y-%m-%d')
             update_next_segment(i, new_end, segments, segments_refined)
 
+    # Pad gradual segments into adjacent flat regions when gradual_padding is set.
+    # Mirrors the abrupt padding pattern: called from within the same module rather
+    # than exposed as a separate top-level step in segments_refine/__init__.py.
+    segments_refined = _pad_gradual_trends(df, segments_refined, method_params)
+
     return segments_refined
+
+
+def _pad_gradual_trends(df: pd.DataFrame, segments: list[dict], method_params: dict) -> list[dict]:
+    """
+    Extends gradual segment end dates by a specified number of days.
+
+    Extends the end date forward, truncating before any non-Flat segment that
+    would be overlapped, and clamping to the last index date. Sets a ``padded``
+    flag on modified segments.
+
+    Args:
+        df (pd.DataFrame): Time series DataFrame.
+        segments (list): List of segment dictionaries.
+        method_params (dict): Supported keys:
+
+            - **gradual_padding** (`int`): Number of days to pad. Defaults to ``0``.
+
+    Returns:
+        list: Segment list with padded gradual boundaries.
+    """
+
+    gradual_padding = method_params.get('gradual_padding', 0)
+    if gradual_padding <= 0:
+        return segments
+
+    segments_padded = deepcopy(segments)
+
+    meta_df = pd.DataFrame(segments)
+    meta_df['start'] = pd.to_datetime(meta_df['start'])
+    meta_df['end'] = pd.to_datetime(meta_df['end'])
+
+    for i, segment in enumerate(segments):
+
+        if segment['direction'] not in ['Up', 'Down'] or segment.get('trend_class') != 'gradual':
+            continue
+
+        gradual_end = pd.to_datetime(segment['end'])
+
+        new_end = gradual_end + pd.Timedelta(days=gradual_padding)
+        overlaps = meta_df.loc[(meta_df['start'] > gradual_end) & (meta_df['start'] <= new_end)]
+        overlaps_nonflats = overlaps[overlaps['direction'] != 'Flat']
+
+        if not overlaps_nonflats.empty:
+            first_notflat_overlap = overlaps_nonflats.iloc[0]
+            new_end = pd.to_datetime(first_notflat_overlap['start']) - pd.Timedelta(days=1)
+
+        new_end = min(new_end, df.index[-1])
+        segments_padded[i]['end'] = new_end.strftime('%Y-%m-%d')
+        update_next_segment(i, new_end, segments, segments_padded)
+
+        segments_padded[i]['padded'] = new_end != gradual_end
+
+    return segments_padded

{pytrendy-1.4.0.dev2 → pytrendy-1.4.0.dev4}/pytrendy/process_signals.py

@@ -53,6 +53,7 @@ def process_signals(df: pd.DataFrame, value_col: str, method_params: dict, debug
 
     THRESHOLD_NOISE = 2.5 # Sensitivity to detecting noise (recommended 0-10)
     THRESHOLD_SMOOTH = 0.001 # Sensitivity to detecting trends as fraction of iqr
+    THRESHOLD_FLAT = 0.835 # Sensitivity to detecting flats as a fraction of min std (non-zero)
 
     # 1. Noise detection via SNR.
     # 1.1 Compute the SNR
@@ -174,31 +175,36 @@ def process_signals(df: pd.DataFrame, value_col: str, method_params: dict, debug
 
     # 3. Flat detection using rolling std of savgol filter.
     # with leading and trailing to cater for periods centred windows doesnt cover
+
+    # 3.1 Compute smoothed signal rolling std to be used by following logic
     df['smoothed'] = savgol_filter(df['value_cleaned'], window_length=WINDOW_SMOOTH, polyorder=1)
     df['smoothed_std'] = df['smoothed'].rolling(WINDOW_FLAT, center=True).std()
     df['smoothed_std_leading'] = df['smoothed'].iloc[::-1].rolling(window=WINDOW_FLAT).std().iloc[::-1]
     df['smoothed_std_trailing'] = df['smoothed'].rolling(WINDOW_FLAT).std()
     df['smoothed_std'] = df['smoothed_std'].fillna(df['smoothed_std_leading']).fillna(df['smoothed_std_trailing'])
 
+    # 3.2 Compute derivative early so flat detection can check both smoothness and motion.
+    derivative_limit = abs(iqr(df[value_col])) * THRESHOLD_SMOOTH
+    df['smoothed_deriv'] = savgol_filter(df[value_col], window_length=WINDOW_SMOOTH, polyorder=1, deriv=1)
+
+    # 3.3 Compute Flat Flag using both std and deriv limits
     df['flat_flag'] = 0
     rolling_std = df['value_cleaned'].rolling(WINDOW_FLAT, center=True).std()
-    min_nonzero_std = rolling_std[rolling_std > 0].min()
-    df.loc[(df['smoothed_std'] <= min_nonzero_std) & (df['noise_flag'] == 0), 'flat_flag'] = 1 
+    nonzero_std = rolling_std[rolling_std > 0]
+    min_nonzero_std = nonzero_std.min() if not nonzero_std.empty else 0.0
+    derivative_near_zero = df['smoothed_deriv'].abs() <= derivative_limit
+    extremely_smooth = df['smoothed_std'] < (min_nonzero_std * THRESHOLD_FLAT)
+    df.loc[(df['smoothed_std'] <= min_nonzero_std) & (df['noise_flag'] == 0) & (derivative_near_zero | extremely_smooth), 'flat_flag'] = 1 
 
-    # 4. Detect up/down trend. Uses first derivates of savgol filter (like diff). 
-    # Savgol filter (rolling avg improvement). Caters for seasonality with tightness to day.
-    # Results in signal that's uptrend > 0, else down. As long as its not on a flat or noise.
+    # 4. Detect up/down trend.
     df['trend_flag'] = 0
     df.loc[df['flat_flag'] == 1, 'trend_flag'] = -2
     df.loc[df['noise_flag'] == 1, 'trend_flag'] = -3
 
-    # Important condition to establish non-trend segments to avoid detecting trends over
-    avoid_condition = (df['flat_flag'] == 0) # flat is always avoided
-    if method_params['avoid_noise']: # noise can be optionally avoided, up to the user
+    avoid_condition = (df['flat_flag'] == 0)
+    if method_params['avoid_noise']:
         avoid_condition &= (df['noise_flag'] == 0)
 
-    derivative_limit = abs(iqr(df[value_col])) * THRESHOLD_SMOOTH
-    df['smoothed_deriv'] = savgol_filter(df[value_col], window_length=WINDOW_SMOOTH, polyorder=1, deriv=1)
     df.loc[(df['smoothed_deriv'] >= derivative_limit) & avoid_condition, 'trend_flag'] = 1
     df.loc[(df['smoothed_deriv'] < -derivative_limit) & avoid_condition, 'trend_flag'] = -1