trend-narrative 0.1.0__tar.gz

trend_narrative-0.1.0/PKG-INFO

@@ -0,0 +1,157 @@
+Metadata-Version: 2.4
+Name: trend-narrative
+Version: 0.1.0
+Summary: Piecewise-linear trend detection and plain-English narrative generation for time-series data.
+Author-email: Yuki Suzuki <yukinko.iwasaki@gmail.com>
+License: MIT
+Keywords: trend,narrative,time-series,piecewise,statistics
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: numpy>=1.24
+Requires-Dist: scipy>=1.10
+Requires-Dist: pwlf>=2.2
+Provides-Extra: dev
+Requires-Dist: pytest>=7.4; extra == "dev"
+Requires-Dist: pytest-cov>=4.1; extra == "dev"
+
+# trend-narrative
+
+A standalone Python package that combines **piecewise-linear trend detection** and **plain-English narrative generation** for time-series data.
+
+Originally developed across two repos:
+- Trend detection logic → `dime-worldbank/mega-boost` (`feature/add_trend_detection_model`)
+- Narrative generation → `dime-worldbank/rpf-country-dash` (`enhancement/trend_narrative`)
+
+---
+
+## Installation
+
+```bash
+pip install -e ".[dev]"   # editable install with dev dependencies
+```
+
+Dependencies: `numpy`, `scipy`, `pwlf`
+
+---
+
+## Quick start
+
+### One-liner
+
+```python
+import numpy as np
+from trend_narrative import generate_narrative
+
+x = np.arange(2010, 2022, dtype=float)
+y = np.array([100, 105, 112, 108, 115, 130, 125, 120, 118, 122, 135, 148], dtype=float)
+
+result = generate_narrative(x, y, metric="total real expenditure")
+print(result["narrative"])
+# → "From 2010 to 2015, the total real expenditure showed an upward trend.
+#    Trend then shifted, reaching a peak in 2015 before reversing into a decline. ..."
+print(f"CV: {result['cv_value']:.1f}%")
+print(result["segments"])
+```
+
+### Step-by-step
+
+```python
+from trend_narrative import InsightExtractor, get_segment_narrative
+
+extractor = InsightExtractor(x, y)
+suite = extractor.extract_full_suite()
+# suite = {"cv_value": 14.2, "segments": [...]}
+
+narrative = get_segment_narrative(
+    suite["segments"],
+    cv_value=suite["cv_value"],
+    metric="health spending",
+)
+print(narrative)
+```
+
+---
+
+## API reference
+
+### `TrendDetector(max_segments=3, threshold=0.05)`
+
+Fits a piecewise-linear model using BIC-optimised segment count and snaps
+breakpoints to integer years / local extrema.
+
+| Method | Returns | Description |
+|---|---|---|
+| `extract_trend(x, y)` | `list[dict]` | Fit model; return per-segment stats |
+| `fit_best_model(x, y)` | `pwlf model \| None` | Run both fitting passes |
+| `calculate_bic(ssr, n, k)` | `float` | Static BIC helper |
+
+Each segment dict contains: `start_year`, `end_year`, `start_value`,
+`end_value`, `slope`, `p_value`.
+
+---
+
+### `InsightExtractor(x, y, detector=None)`
+
+High-level facade combining volatility and trend detection.
+
+| Method | Returns | Description |
+|---|---|---|
+| `get_volatility()` | `float` | Coefficient of Variation (%) |
+| `get_structural_segments()` | `list[dict]` | Delegates to `TrendDetector` |
+| `extract_full_suite()` | `dict` | `{cv_value, segments}` |
+
+---
+
+### `get_segment_narrative(segments, cv_value, metric="expenditure")`
+
+Convert segment data into a multi-sentence English narrative.
+
+- No segments + low CV → *"remained highly stable"*
+- No segments + high CV → *"exhibited significant volatility"*
+- Single segment → direction + % change sentence
+- Multi-segment → transition phrases (peak / trough / continuation)
+
+---
+
+### `consolidate_segments(segments)`
+
+Merge consecutive segments sharing the same slope direction. Useful for
+simplifying noisy multi-segment fits before narrative generation.
+
+---
+
+### `millify(n)`
+
+Format large numbers with suffix: `1_500_000 → "1.50 M"`.
+
+---
+
+## Running tests
+
+```bash
+pytest
+# or with coverage:
+pytest --cov=trend_narrative --cov-report=term-missing
+```
+
+---
+
+## Project structure
+
+```
+trend-narrative/
+├── trend_narrative/
+│   ├── __init__.py        # Public API
+│   ├── detector.py        # TrendDetector – piecewise-linear fitting
+│   ├── extractor.py       # InsightExtractor – volatility + trend facade
+│   └── narrative.py       # Narrative generation + millify helper
+├── tests/
+│   ├── test_detector.py
+│   ├── test_extractor.py
+│   └── test_narrative.py
+├── pyproject.toml
+└── README.md
+```

trend_narrative-0.1.0/README.md

@@ -0,0 +1,138 @@
+# trend-narrative
+
+A standalone Python package that combines **piecewise-linear trend detection** and **plain-English narrative generation** for time-series data.
+
+Originally developed across two repos:
+- Trend detection logic → `dime-worldbank/mega-boost` (`feature/add_trend_detection_model`)
+- Narrative generation → `dime-worldbank/rpf-country-dash` (`enhancement/trend_narrative`)
+
+---
+
+## Installation
+
+```bash
+pip install -e ".[dev]"   # editable install with dev dependencies
+```
+
+Dependencies: `numpy`, `scipy`, `pwlf`
+
+---
+
+## Quick start
+
+### One-liner
+
+```python
+import numpy as np
+from trend_narrative import generate_narrative
+
+x = np.arange(2010, 2022, dtype=float)
+y = np.array([100, 105, 112, 108, 115, 130, 125, 120, 118, 122, 135, 148], dtype=float)
+
+result = generate_narrative(x, y, metric="total real expenditure")
+print(result["narrative"])
+# → "From 2010 to 2015, the total real expenditure showed an upward trend.
+#    Trend then shifted, reaching a peak in 2015 before reversing into a decline. ..."
+print(f"CV: {result['cv_value']:.1f}%")
+print(result["segments"])
+```
+
+### Step-by-step
+
+```python
+from trend_narrative import InsightExtractor, get_segment_narrative
+
+extractor = InsightExtractor(x, y)
+suite = extractor.extract_full_suite()
+# suite = {"cv_value": 14.2, "segments": [...]}
+
+narrative = get_segment_narrative(
+    suite["segments"],
+    cv_value=suite["cv_value"],
+    metric="health spending",
+)
+print(narrative)
+```
+
+---
+
+## API reference
+
+### `TrendDetector(max_segments=3, threshold=0.05)`
+
+Fits a piecewise-linear model using BIC-optimised segment count and snaps
+breakpoints to integer years / local extrema.
+
+| Method | Returns | Description |
+|---|---|---|
+| `extract_trend(x, y)` | `list[dict]` | Fit model; return per-segment stats |
+| `fit_best_model(x, y)` | `pwlf model \| None` | Run both fitting passes |
+| `calculate_bic(ssr, n, k)` | `float` | Static BIC helper |
+
+Each segment dict contains: `start_year`, `end_year`, `start_value`,
+`end_value`, `slope`, `p_value`.
+
+---
+
+### `InsightExtractor(x, y, detector=None)`
+
+High-level facade combining volatility and trend detection.
+
+| Method | Returns | Description |
+|---|---|---|
+| `get_volatility()` | `float` | Coefficient of Variation (%) |
+| `get_structural_segments()` | `list[dict]` | Delegates to `TrendDetector` |
+| `extract_full_suite()` | `dict` | `{cv_value, segments}` |
+
+---
+
+### `get_segment_narrative(segments, cv_value, metric="expenditure")`
+
+Convert segment data into a multi-sentence English narrative.
+
+- No segments + low CV → *"remained highly stable"*
+- No segments + high CV → *"exhibited significant volatility"*
+- Single segment → direction + % change sentence
+- Multi-segment → transition phrases (peak / trough / continuation)
+
+---
+
+### `consolidate_segments(segments)`
+
+Merge consecutive segments sharing the same slope direction. Useful for
+simplifying noisy multi-segment fits before narrative generation.
+
+---
+
+### `millify(n)`
+
+Format large numbers with suffix: `1_500_000 → "1.50 M"`.
+
+---
+
+## Running tests
+
+```bash
+pytest
+# or with coverage:
+pytest --cov=trend_narrative --cov-report=term-missing
+```
+
+---
+
+## Project structure
+
+```
+trend-narrative/
+├── trend_narrative/
+│   ├── __init__.py        # Public API
+│   ├── detector.py        # TrendDetector – piecewise-linear fitting
+│   ├── extractor.py       # InsightExtractor – volatility + trend facade
+│   └── narrative.py       # Narrative generation + millify helper
+├── tests/
+│   ├── test_detector.py
+│   ├── test_extractor.py
+│   └── test_narrative.py
+├── pyproject.toml
+└── README.md
+```

trend_narrative-0.1.0/pyproject.toml

@@ -0,0 +1,40 @@
+[build-system]
+requires = ["setuptools>=42", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "trend-narrative"
+version = "0.1.0"
+description = "Piecewise-linear trend detection and plain-English narrative generation for time-series data."
+readme = "README.md"
+requires-python = ">=3.9"
+license = { text = "MIT" }
+authors = [
+    { name = "Yuki Suzuki", email = "yukinko.iwasaki@gmail.com" }
+]
+keywords = ["trend", "narrative", "time-series", "piecewise", "statistics"]
+classifiers = [
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+]
+
+dependencies = [
+    "numpy>=1.24",
+    "scipy>=1.10",
+    "pwlf>=2.2",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=7.4",
+    "pytest-cov>=4.1",
+]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-v --tb=short"
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["trend_narrative*"]

trend_narrative-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

trend_narrative-0.1.0/tests/test_detector.py

@@ -0,0 +1,162 @@
+"""Unit tests for trend_narrative.detector.TrendDetector."""
+
+import numpy as np
+import pytest
+
+from trend_narrative.detector import TrendDetector
+
+
+# ---------------------------------------------------------------------------
+# Fixtures
+# ---------------------------------------------------------------------------
+
+@pytest.fixture
+def linear_up():
+    """Strictly increasing linear series (2010–2021)."""
+    x = np.arange(2010, 2022, dtype=float)
+    y = 100.0 + 5.0 * (x - 2010)
+    return x, y
+
+
+@pytest.fixture
+def linear_down():
+    """Strictly decreasing linear series."""
+    x = np.arange(2010, 2022, dtype=float)
+    y = 200.0 - 5.0 * (x - 2010)
+    return x, y
+
+
+@pytest.fixture
+def v_shape():
+    """A V-shape with a clear trough in the middle."""
+    x = np.arange(2005, 2021, dtype=float)
+    y = np.concatenate([
+        100.0 - 4.0 * np.arange(8),   # decline 2005-2012
+        68.0  + 4.0 * np.arange(8),   # recovery 2013-2020
+    ])
+    return x, y
+
+
+@pytest.fixture
+def flat():
+    """Nearly flat series – no real trend."""
+    rng = np.random.default_rng(0)
+    x = np.arange(2010, 2022, dtype=float)
+    y = 100.0 + rng.normal(0, 0.5, len(x))   # tiny noise
+    return x, y
+
+
+# ---------------------------------------------------------------------------
+# calculate_bic
+# ---------------------------------------------------------------------------
+
+class TestCalculateBic:
+    def test_lower_ssr_gives_lower_bic(self):
+        bic_good = TrendDetector.calculate_bic(ssr=10, n_data_points=20, n_segments=2)
+        bic_bad  = TrendDetector.calculate_bic(ssr=100, n_data_points=20, n_segments=2)
+        assert bic_good < bic_bad
+
+    def test_more_segments_penalises_bic(self):
+        """Extra segments should increase BIC when SSR is held constant."""
+        bic_simple  = TrendDetector.calculate_bic(ssr=50, n_data_points=20, n_segments=1)
+        bic_complex = TrendDetector.calculate_bic(ssr=50, n_data_points=20, n_segments=3)
+        assert bic_complex > bic_simple
+
+    def test_returns_float(self):
+        result = TrendDetector.calculate_bic(100, 20, 2)
+        assert isinstance(result, float)
+
+
+# ---------------------------------------------------------------------------
+# find_local_maxima_years
+# ---------------------------------------------------------------------------
+
+class TestFindLocalMaximaYears:
+    def test_detects_peak(self):
+        detector = TrendDetector()
+        x = np.array([2010, 2011, 2012, 2013, 2014], dtype=float)
+        y = np.array([1, 3, 5, 3, 1], dtype=float)   # peak at 2012
+        extrema = detector.find_local_maxima_years(x, y)
+        assert 2012.0 in extrema
+
+    def test_detects_valley(self):
+        detector = TrendDetector()
+        x = np.array([2010, 2011, 2012, 2013, 2014], dtype=float)
+        y = np.array([5, 3, 1, 3, 5], dtype=float)   # valley at 2012
+        extrema = detector.find_local_maxima_years(x, y)
+        assert 2012.0 in extrema
+
+    def test_monotone_has_no_extrema(self):
+        detector = TrendDetector()
+        x = np.arange(2010, 2016, dtype=float)
+        y = np.arange(6, dtype=float)
+        extrema = detector.find_local_maxima_years(x, y)
+        assert extrema == []
+
+
+# ---------------------------------------------------------------------------
+# extract_trend – end-to-end
+# ---------------------------------------------------------------------------
+
+class TestExtractTrend:
+    def test_returns_list(self, linear_up):
+        x, y = linear_up
+        detector = TrendDetector()
+        result = detector.extract_trend(x, y)
+        assert isinstance(result, list)
+
+    def test_segment_keys_present(self, linear_up):
+        x, y = linear_up
+        detector = TrendDetector()
+        result = detector.extract_trend(x, y)
+        if result:
+            expected_keys = {"start_year", "end_year", "start_value", "end_value", "slope", "p_value"}
+            assert expected_keys.issubset(result[0].keys())
+
+    def test_upward_trend_positive_slope(self, linear_up):
+        x, y = linear_up
+        detector = TrendDetector()
+        result = detector.extract_trend(x, y)
+        if result:
+            assert all(seg["slope"] > 0 for seg in result)
+
+    def test_downward_trend_negative_slope(self, linear_down):
+        x, y = linear_down
+        detector = TrendDetector()
+        result = detector.extract_trend(x, y)
+        if result:
+            assert all(seg["slope"] < 0 for seg in result)
+
+    def test_v_shape_returns_two_segments(self, v_shape):
+        x, y = v_shape
+        detector = TrendDetector()
+        result = detector.extract_trend(x, y)
+        # Allow 0 (no valid fit) or 2 (correct detection)
+        assert len(result) in (0, 2)
+        if len(result) == 2:
+            assert result[0]["slope"] < 0   # declining first
+            assert result[1]["slope"] > 0   # recovering second
+
+    def test_accepts_list_input(self):
+        x = list(range(2010, 2022))
+        y = [float(i) * 3 for i in range(12)]
+        result = TrendDetector().extract_trend(x, y)
+        assert isinstance(result, list)
+
+    def test_segments_are_contiguous(self, v_shape):
+        x, y = v_shape
+        result = TrendDetector().extract_trend(x, y)
+        for i in range(1, len(result)):
+            assert result[i]["start_year"] == result[i - 1]["end_year"]
+
+    def test_too_few_points_returns_empty(self):
+        x = np.array([2010, 2011], dtype=float)
+        y = np.array([100, 110], dtype=float)
+        result = TrendDetector().extract_trend(x, y)
+        assert result == []
+
+    def test_max_segments_respected(self, v_shape):
+        x, y = v_shape
+        detector = TrendDetector(max_segments=1)
+        result = detector.extract_trend(x, y)
+        assert len(result) <= 1

trend_narrative-0.1.0/tests/test_extractor.py

@@ -0,0 +1,121 @@
+"""Unit tests for trend_narrative.extractor.InsightExtractor."""
+
+import math
+
+import numpy as np
+import pytest
+
+from trend_narrative.extractor import InsightExtractor
+
+
+# ---------------------------------------------------------------------------
+# Fixtures
+# ---------------------------------------------------------------------------
+
+@pytest.fixture
+def stable_series():
+    x = np.arange(2010, 2022, dtype=float)
+    rng = np.random.default_rng(1)
+    y = 100.0 + rng.normal(0, 1, len(x))   # CV ≈ 1 %
+    return x, y
+
+
+@pytest.fixture
+def volatile_series():
+    rng = np.random.default_rng(42)
+    x = np.arange(2000, 2020, dtype=float)
+    y = rng.uniform(10, 100, len(x))        # high CV
+    return x, y
+
+
+@pytest.fixture
+def trending_series():
+    x = np.arange(2010, 2022, dtype=float)
+    y = 100.0 + 10.0 * (x - 2010)
+    return x, y
+
+
+# ---------------------------------------------------------------------------
+# get_volatility
+# ---------------------------------------------------------------------------
+
+class TestGetVolatility:
+    def test_stable_series_low_cv(self, stable_series):
+        x, y = stable_series
+        cv = InsightExtractor(x, y).get_volatility()
+        assert cv < 5.0
+
+    def test_volatile_series_high_cv(self, volatile_series):
+        x, y = volatile_series
+        cv = InsightExtractor(x, y).get_volatility()
+        assert cv > 15.0
+
+    def test_cv_is_float(self, stable_series):
+        x, y = stable_series
+        cv = InsightExtractor(x, y).get_volatility()
+        assert isinstance(cv, float)
+
+    def test_zero_mean_returns_nan(self):
+        x = np.arange(5, dtype=float)
+        y = np.zeros(5)
+        cv = InsightExtractor(x, y).get_volatility()
+        assert math.isnan(cv)
+
+    def test_constant_series_zero_cv(self):
+        x = np.arange(2010, 2022, dtype=float)
+        y = np.full(12, 50.0)
+        cv = InsightExtractor(x, y).get_volatility()
+        assert cv == pytest.approx(0.0)
+
+
+# ---------------------------------------------------------------------------
+# get_structural_segments
+# ---------------------------------------------------------------------------
+
+class TestGetStructuralSegments:
+    def test_returns_list(self, trending_series):
+        x, y = trending_series
+        segments = InsightExtractor(x, y).get_structural_segments()
+        assert isinstance(segments, list)
+
+    def test_trending_series_segments(self, trending_series):
+        x, y = trending_series
+        segments = InsightExtractor(x, y).get_structural_segments()
+        # May be empty (no valid fit) or list of dicts
+        assert all(isinstance(s, dict) for s in segments)
+
+
+# ---------------------------------------------------------------------------
+# extract_full_suite
+# ---------------------------------------------------------------------------
+
+class TestExtractFullSuite:
+    def test_keys_present(self, stable_series):
+        x, y = stable_series
+        result = InsightExtractor(x, y).extract_full_suite()
+        assert "cv_value" in result
+        assert "segments" in result
+
+    def test_cv_value_is_numeric(self, stable_series):
+        x, y = stable_series
+        result = InsightExtractor(x, y).extract_full_suite()
+        assert isinstance(result["cv_value"], float)
+
+    def test_segments_is_list(self, stable_series):
+        x, y = stable_series
+        result = InsightExtractor(x, y).extract_full_suite()
+        assert isinstance(result["segments"], list)
+
+    def test_custom_detector_used(self, trending_series):
+        """Passing a max_segments=1 detector limits segment count."""
+        from trend_narrative.detector import TrendDetector
+        x, y = trending_series
+        detector = TrendDetector(max_segments=1)
+        result = InsightExtractor(x, y, detector=detector).extract_full_suite()
+        assert len(result["segments"]) <= 1
+
+    def test_array_like_inputs_accepted(self):
+        x = list(range(2010, 2022))
+        y = [float(i ** 2) for i in range(12)]
+        result = InsightExtractor(x, y).extract_full_suite()
+        assert "cv_value" in result