processbehavior 0.2.1__tar.gz → 0.3.1__tar.gz

{processbehavior-0.2.1 → processbehavior-0.3.1}/CHANGELOG.md

@@ -7,6 +7,97 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.3.1] - 2026-09-14
+
+### Fixed
+- **Derived variables: ``evaluate`` and ``validate`` now keep their never-raises promise.**
+  An adversarial pass (36,000 fuzzed evaluations) found four ways to make them raise from
+  pandas: ±inf in the source column (``equal_freq`` / ``equal_width``: "bins must increase
+  monotonically"), range labels that collide at six significant digits on a tiny range or a
+  near-constant ``sd`` fit ("labels must be unique"), duplicate or null explicit
+  ``bin_labels``, and a string ``shift`` / ``exponent`` reaching numpy. Now: ±inf leaves the
+  bin fit, becomes NaN, and is counted in ``n_invalid`` with a message; range labels gain
+  precision until distinct (numbered bins as the last resort, with a message); explicit
+  labels must be non-empty, null-free and unique; ``shift``, ``exponent`` and ``breaks`` must
+  be finite numbers; ``on_invalid`` must be ``'error'`` or ``'na'``; ``n`` must be a positive
+  integer (numpy integers accepted, booleans rejected) — all at construction, as
+  ``ValidationError``. A datetime or categorical source now evaluates to an all-NA result
+  with a message instead of transforming nanoseconds or category codes. A permanent seeded
+  fuzz test guards the contract.
+
+### Changed
+- **Every non-finite transform result is a domain violation.** ``log``/``sqrt`` of ``inf``,
+  ``square`` overflow and ``inverse`` of a denormal used to return ``inf`` with
+  ``n_invalid=0`` (only ``power`` overflow was flagged); all are now violations, so
+  ``on_invalid`` applies and an ``inf`` can no longer reach the limits. A z-score of a
+  constant (or single-value) column is now a violation for every value rather than a silent
+  all-NaN column, so ``on_invalid='error'`` names the derivation at ``formulate``.
+  ``arcsin`` on values in (1, 100] adds the hint "if these are percentages, divide by 100".
+  Requesting more equal-frequency / equal-width bins than there are distinct values now fits
+  one bin per distinct value and says so. ``'ordinal'`` labels above five bins say they fell
+  back to numbered bins. ``to_dict`` converts numpy scalars and arrays (it was documented
+  JSON-safe but was not); ``from_dict`` decodes the ``Infinity``/``NaN`` tags only under
+  numeric keys, so a bin labelled ``"NaN"`` stays a string; a missing ``id`` is a
+  ``ValidationError``, not a ``KeyError``. Finite inputs with distinct labels produce
+  byte-identical edges, labels and values.
+
+## [0.3.0] - 2026-09-14
+
+### Added
+- **The design report states where sigma comes from at the observed structure and
+  series length.** One line under ``Structure``: ``Series length: T=4. Sigma for X/mR
+  rests on 3 moving ranges.`` With every cell replicated it reads ``Series length: T=1.
+  Sigma rests on within-cell replication.`` — a single time period is a complete study.
+  Exposed as ``study.series_length`` (``SeriesLengthPrecision``, frozen) and
+  ``study.series_length_description``. The result also carries ``within_cell_df`` and,
+  for the moving-range case, ``mr_interval_80`` — the p10/p90 of MRbar/d2 by n = 3..30
+  from ``MR_SIGMA_INTERVAL_80`` (``spc_constants``), generated by
+  ``validation/short_series_bands.py`` (contributed by @rabujamra in #114 / #119) — for
+  callers who want the numbers; the report prints only the sentence. It is a fact beside
+  the design state, not a judgment on it: no threshold, label, or warning, and the design
+  state, recommended chart and analysis menu do not move with T (pinned by test). Closes
+  the series-length half of #114.
+
+### Changed
+- **The R2 method for partial replication is reported as ``ma2``, not ``hybrid``.** The
+  docs described a per-cell hybrid for design state 3 (exact deviation where a cell has
+  n ≥ 2, moving average where n = 1) and said it was validated by Monte Carlo. The code
+  has not done that since cb037f8 (2026-03-05): with any singleton cell, R2 is Bishop's
+  ungrouped 2-point moving average over the full canonical sequence (Eq 13.7–13.9) for
+  every observation, and the 89 ADS 3 reference assertions hold against that. Only the word
+  survived, in six docs pages (three of which described it three different ways), the
+  detector's method token, the analysis-plan limitation text, two docstrings and two test
+  names. All now say what the code does. ``R2Method`` is ``Literal['exact', 'ma2']``;
+  ``SDSRegistry.get_r2_method`` returns ``'ma2'`` where it returned ``'hybrid'``;
+  ``get_sds_characteristics`` and ``SDSAnalysisPlan.residual_calculation_method`` use the
+  same two spellings in place of ``'within_cell'``, ``'moving_average'`` and ``'hybrid'``.
+  No arithmetic changed; none of these names is exported from the package top level.
+
+### Dependencies
+- ``plotly>=5.18,<8`` (was ``<7``). plotly 7.0.0 (2026-08-25) was tested against the full
+  suite before the bound moved: its breaking changes (mapbox traces, decimal-fraction
+  ``rgb()`` strings, ``hsv()``, Kaleido < 1.0, figure_factory) touch nothing here. #116.
+
+### Fixed
+- **A missing value in a factor column no longer makes a two-factor study fail.** With one
+  factor, rows with a blank factor value were dropped silently; with two or more, the
+  composite subgroup label was built before that drop and ``formulate`` raised
+  ``ValidationError: Cannot build RSG ... missing values in factor columns``. The same
+  file formulated with either factor alone and failed with both (a 85,101-row survey
+  file with 7,980 blank ``SURVEY QUESTION`` rows). Rows with no value in any factor
+  column now leave the analysis before the label is built, for any number of factors,
+  and the library says so with a ``ProcessBehaviorWarning`` naming the count per
+  factor. Response and time missing values are handled as before.
+- **``evaluate`` no longer raises on a constant column.** Binning a column with no spread
+  (every value identical) returned the documented no-spread result for ``equal_freq`` but
+  raised ``ValueError: Bin edges must be unique`` from ``pd.cut`` for ``equal_width`` and
+  ``sd``, because their fitted edges were identical yet finite and slipped past the guard.
+  The derivations module promises ``evaluate`` never raises on a routine state; a constant
+  column is one. All three methods now return the same no-spread result, whose ``fitted``
+  dict also carries ``n_bins: 0``, ``edges: []`` and ``labels: []`` so callers see one
+  shape. Explicit ``breaks`` on a constant column still bin. Found through the app on a
+  single-time-point file whose TIME column was a constant.
+
 ## [0.2.1] - 2026-09-09
 
 ### Fixed
@@ -812,7 +903,9 @@ and may change between minor versions until 1.0.
 - `natsort >= 8.0`
 - `plotly >= 5.18, < 7`
 
-[Unreleased]: https://github.com/cnicholas/processbehavior/compare/v0.2.1...HEAD
+[Unreleased]: https://github.com/cnicholas/processbehavior/compare/v0.3.1...HEAD
+[0.3.1]: https://github.com/cnicholas/processbehavior/compare/v0.3.0...v0.3.1
+[0.3.0]: https://github.com/cnicholas/processbehavior/compare/v0.2.1...v0.3.0
 [0.2.1]: https://github.com/cnicholas/processbehavior/compare/v0.2.0...v0.2.1
 [0.2.0]: https://github.com/cnicholas/processbehavior/compare/v0.1.1...v0.2.0
 [0.1.1]: https://github.com/cnicholas/processbehavior/compare/v0.1.0...v0.1.1

{processbehavior-0.2.1 → processbehavior-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: processbehavior
-Version: 0.2.1
+Version: 0.3.1
 Summary: Python-native Process Behavior & Control Charts (spec-driven SPC with tidy outputs)
 Project-URL: Homepage, https://github.com/cnicholas/processbehavior
 Project-URL: Repository, https://github.com/cnicholas/processbehavior
@@ -28,7 +28,7 @@ Requires-Python: >=3.10
 Requires-Dist: natsort>=8.0.0
 Requires-Dist: numpy>=1.23
 Requires-Dist: pandas<3,>=2.0
-Requires-Dist: plotly<7,>=5.18
+Requires-Dist: plotly<8,>=5.18
 Requires-Dist: typing-extensions>=4.0; python_version < '3.11'
 Provides-Extra: dev
 Requires-Dist: filelock>=3.20.3; extra == 'dev'

{processbehavior-0.2.1 → processbehavior-0.3.1}/processbehavior/__init__.py

@@ -47,7 +47,7 @@ its ``.sds`` field. Internally some modules still use the legacy "SDS"
 prefix (e.g. ``sds_detector.py``); the model itself is three-state.
 """
 
-__version__ = '0.2.1'
+__version__ = '0.3.1'
 
 # Result object
 from processbehavior.analysis_result import AnalysisResult

{processbehavior-0.2.1 → processbehavior-0.3.1}/processbehavior/analysis_dataset.py

@@ -183,7 +183,7 @@ class AnalysisDataSet:
             - n_cell_min: minimum observations per cell
             - n_cell_max: maximum observations per cell
             - K_obs: number of unique factor levels observed
-            - r2_method: 'exact', 'ma2', or 'hybrid'
+            - r2_method: 'exact' or 'ma2'
             Returns None if structure stats haven't been computed.
         """
         if self._structure_stats is None:

{processbehavior-0.2.1 → processbehavior-0.3.1}/processbehavior/data_preparation.py

@@ -16,6 +16,7 @@ Follows the Pythonic Hadley philosophy:
 from __future__ import annotations
 
 import logging
+import warnings
 from functools import reduce
 from typing import TYPE_CHECKING
 
@@ -24,7 +25,7 @@ import pandas as pd
 from natsort import natsorted
 from pandas.api.types import is_numeric_dtype
 
-from .exceptions import ColumnNotFoundError, FactorNotFoundError, ValidationError
+from .exceptions import ColumnNotFoundError, FactorNotFoundError, ProcessBehaviorWarning, ValidationError
 
 if TYPE_CHECKING:
     from .formulation_spec import FormulationSpec
@@ -281,6 +282,13 @@ class DataPreparation:
 
         # Add composite grouping variable if needed
         if spec.has_grouping:
+            # A row with no value in a factor column belongs to no cell, so it leaves the
+            # analysis here — for one factor or many. The single-factor path used to drop
+            # such rows silently via the rsg dropna below; the multi-factor path raised
+            # from _add_composite_column before reaching it, so the same file formulated
+            # with one factor and failed with two (Tom's survey file, 7,980 blank
+            # SURVEY QUESTION rows). Both paths now drop first and say so.
+            out = self._drop_rows_missing_factor_values(out, spec)
             out = self._add_grouping_column(out, spec)
 
             # Drop rows with missing values in analysis-critical columns BEFORE
@@ -533,6 +541,30 @@ class DataPreparation:
     # Private Helper Methods
     # ========================================================================
 
+    def _drop_rows_missing_factor_values(self, df: pd.DataFrame, spec: FormulationSpec) -> pd.DataFrame:
+        """
+        Drop rows with a missing value in any factor column, and report it.
+
+        Reported as a ``ProcessBehaviorWarning`` (the house style for "the
+        library changed your data on your behalf"), naming the count per
+        factor, so an analyst sees that 7,980 rows left the study rather than
+        discovering a smaller N later.
+        """
+        factor_cols = spec.rsg_vars_list
+        missing = df[factor_cols].isna()
+        if not missing.any().any():
+            return df
+        per_col = {col: int(n) for col, n in missing.sum().items() if n}
+        n_rows = int(missing.any(axis=1).sum())
+        warnings.warn(
+            f'Dropped {n_rows:,} of {len(df):,} rows with no value in a factor column: '
+            + ', '.join(f'{col} ({n:,} missing)' for col, n in per_col.items())
+            + '. A row with no factor value belongs to no cell, so it is excluded from the analysis.',
+            ProcessBehaviorWarning,
+            stacklevel=6,
+        )
+        return df.loc[~missing.any(axis=1)]
+
     def _add_grouping_column(self, df: pd.DataFrame, spec: FormulationSpec) -> pd.DataFrame:
         """
         Add composite grouping column (e.g., 'lane_head').

{processbehavior-0.2.1 → processbehavior-0.3.1}/processbehavior/datasets/synthetic.py

@@ -416,13 +416,13 @@ def make_sds3(  # noqa: C901
     Characteristics:
     ---------------
     - Some cells have n=1, others have n≥2
-    - Requires hybrid R2 estimation
+    - Mixed cell sizes: R2 by MA2 across the full sequence
     - Reflects real-world data collection reality
     - Most challenging to analyze correctly
 
     Use Cases:
     ---------
-    - Testing hybrid variance estimation algorithms
+    - Testing R2 selection with mixed cell sizes
     - Demonstrating real-world data structures
     - Validating robustness to missing data
     - Development of SDS detection logic