kalmanflow 0.1.0a1__tar.gz

kalmanflow-0.1.0a1/Documentation/API_REFERENCE.md

@@ -0,0 +1,107 @@
+# API reference
+
+This page covers the complete supported package surface in KalmanFlow 0.1.0.
+Import application-facing names from `kalmanflow`. The advanced types below are
+also package exports, but are intended for custom models and integrations.
+Names in modules beginning with `_` are internal and may change without a
+compatibility promise.
+
+`kalmanflow.__version__` reports the installed package version.
+
+## Reservoir estimation
+
+| Export | Purpose |
+| --- | --- |
+| `Observation(timestamp, storage, discharge)` | Immutable streaming input. Timestamp must be timezone-aware and no finer than microsecond precision; use `NaN` for a missing value after initialization. |
+| `OnlineReservoirInflow` | Default acre-ft/cfs streaming estimator. Construct with the five scalar diagonal noise values; use `process`, `process_many`, `initialized`, and `pending_count`. |
+| `OnlineReservoirInflow.from_config(config)` | Creates a stream from reviewed configuration, including its units and smoothing lag. |
+| `OnlineReservoirInflow.from_checkpoint(checkpoint, config=...)` | Restores a configured stream. The reservoir ID and fingerprinted model, covariance, lag, and unit settings must match. |
+| `OnlineReservoirInflow.checkpoint()` | Produces resumable state after a successful call. It requires a reservoir ID; streams made with `from_config` have one. |
+| `ReservoirFlowEstimate` | One timestamped inflow value with `prediction_flag` and `smoothing_flag`. |
+| `ReservoirFlowUpdate` | The streaming return value: `filtered_inflows` are causal and `revised_inflows` are absolute, finalized replacements. |
+| `get_reservoir_inflow(storage, outflow, ...)` | Default acre-ft/cfs batch estimator. It returns the six documented inflow and provenance columns. |
+| `get_reservoir_inflow_from_config(storage, outflow, config)` | Batch estimator using a `ReservoirConfig` and its unit system. |
+| `run_inflow_model(observations, ...)` | DataFrame convenience form of `get_reservoir_inflow`; the frame must have `storage` and `outflow` columns. |
+| `OutputFlag` | `NORMAL` or `PREDICTED` describes observation completeness; `SMOOTHED` or `NON_SMOOTHED` describes estimate provenance. |
+
+Both batch series must share an exactly equal timezone-aware, strictly
+increasing index with at most microsecond timestamp precision. The package
+neither aligns nor cleans inputs. A revised
+inflow always replaces the causal value at the same timestamp; it is never a
+delta. See [model behavior](INFLOW_MODEL_BEHAVIOR.md) for initialization,
+partial observations, and output columns.
+
+The reservoir inflow state is an inferred net balance contribution from the
+supplied storage and accounted outflow. Measured outflow should cover outlet
+releases, spills, and outward diversions or withdrawals as applicable.
+Precipitation, evaporation, seepage, and other water exchanges are not
+separate API terms. The linear-Gaussian estimate is not constrained to be
+nonnegative; sensor bias, storage-datum changes, and rating-curve changes can
+also appear as model mismatch.
+
+## Configuration and units
+
+| Export | Purpose |
+| --- | --- |
+| `ReservoirConfig` | Immutable reservoir identity, covariance, lag, units, version, and metadata. `q`, `r`, and `p0` are 3×3, 2×2, and 3×3 covariance matrices respectively. |
+| `InitializationStrategy` | Initialization policy enum. The currently supported value is `FIRST_TWO_VALID_STORAGE`. |
+| `InflowUnits` | Metadata enum: `CUBIC_FEET_PER_SECOND` or `SYSTEM_FLOW_RATE`. CFS requires a unit system whose flow label is `cfs`. |
+| `UnitSystem` | Volume/flow labels and rate-to-volume-per-second conversion. Use `us_customary()`, `si()`, `flow_to_volume()`, and `volume_to_flow_rate()` as needed. |
+| `CFS_TO_ACRE_FEET_PER_SECOND` | Conversion constant used by the default acre-ft/cfs model. |
+
+`q` is continuous-time diffusion covariance, not a discrete fixed-interval Q
+matrix. All covariance inputs must be finite, symmetric, and positive
+semidefinite; `r` must have a strictly positive diagonal. See
+[configuration](CONFIGURATION.md) for a complete example.
+
+## Bayesian noise evaluation and tuning
+
+The Bayesian search API is **experimental** and requires
+`pip install "kalmanflow[tuning]"`. Its API, selection rules, and result schema may
+change between releases. Review and independently validate any proposed
+configuration. `evaluate_configuration` is available with the base installation;
+the SciPy/scikit-learn optimizer dependencies load only when search is invoked.
+
+| Export | Purpose |
+| --- | --- |
+| `ValidationWindow(name, start, end, weight=None)` | Named, timezone-aware half-open interval `[start, end)` for causal scoring. Window boundaries may use nanosecond precision even though scored observation timestamps are limited to microseconds. |
+| `BayesianEvaluationSettings` | Shared warmup, innovation, calibration-warning, and numerical-stability settings. |
+| `BayesianTuningSettings` | Trial counts, covariance multiplier bounds, acquisition settings, and optional upstream-proxy shape/timing gate. |
+| `tune_inflow_noise_bayesian(...)` | Proposes five diagonal `q`/`r` terms from causal innovation scores. It needs at least three windows and a new proposed configuration version. |
+| `BayesianTuningResult` | Proposed `selected_config`, parameters, trial summary, selected-window diagnostics, optional proxy diagnostics, warnings, and timing. |
+| `evaluate_configuration(...)` | Scores one already-frozen configuration without searching or modifying it. |
+| `ConfigEvaluationResult` | Frozen configuration, compact candidate summary, window diagnostics, and warnings. |
+| `BayesianTuningError` | Raised when no candidate meets the tuning workflow's hard requirements. |
+
+Tuning requires diagonal `q`, `r`, and `p0`, clean matched pandas series, at
+least three positive inflow-increment seeds, and at least three distinct,
+non-overlapping windows. Every seed is evaluated, so `total_trials` must be at
+least the number of seeds. An optional upstream series is a shape/timing
+diagnostic only, not a total-inflow label or Kalman observation. See
+[Bayesian noise tuning](BAYESIAN_NOISE_TUNING.md) for selection and report
+details.
+
+## Advanced state-space interfaces
+
+These exports make it possible to use KalmanFlow's filtering and smoothing with
+a different state-space model. They operate on NumPy arrays and do not impose
+reservoir units.
+
+| Export | Purpose |
+| --- | --- |
+| `kalman_filter(...)` | Batch linear-Gaussian filter. Transition, process, offset, and control arrays may be shared or have exactly `n_times - 1` entries; observation arrays may be shared or have exactly `n_times` entries. Model arrays must be finite, covariance arrays symmetric positive semidefinite, and `NaN` is the only omitted observation value. |
+| `predict_state(...)` | Predicts one state mean and covariance, optionally with a control offset. |
+| `initial_filter_step(...)` / `kalman_step(...)` | Create immutable timestamped filter records for the initial or a later update. |
+| `FilterStep` / `KalmanFilterResult` | Filter records and complete batch outputs, including innovations, innovation covariance, update mask, transitions, and log likelihood. |
+| `smooth_filter_steps(steps)` | Runs full-record RTS smoothing and returns one `SmoothedStep` per input step. Empty input returns an empty tuple. |
+| `OnlineFixedLagRTS` | Time-lagged RTS smoother with `add_step`, `provisional`, active-step inspection, and a bounded memory window. |
+| `SmoothedStep` | Immutable smoothed timestamp, state mean/covariance, and provenance flags. |
+| `StateSpaceModel` / `ReservoirStateSpaceModel` | Protocol and physical three-state reservoir implementation. The reservoir state order is `[storage, inflow_rate, true_outflow_rate]`. |
+| `ReservoirBackend` | Connects `ReservoirStateSpaceModel` to the generic pipeline. |
+| `OnlineInflowPipeline` / `PipelineUpdate` | Generic ordered-stream coordinator and its per-call results. Use when supplying a compatible custom backend and smoother. |
+
+`OnlineFixedLagRTS` releases a state once the elapsed-time lag has passed and
+raises `OverflowError` if its active window reaches `max_window_steps` before
+release. Generic pipeline state and protocol types are available from
+`kalmanflow.pipeline` for custom checkpoint/replay implementations, but the
+reservoir checkpoint byte format remains internal.

kalmanflow-0.1.0a1/Documentation/ARCHITECTURE.md

@@ -0,0 +1,83 @@
+# KalmanFlow architecture
+
+`kalmanflow` is a Python package for estimating reservoir inflow from measured storage and discharge. It implements a three-state, linear-Gaussian water-balance model plus a causal Kalman filter and fixed-lag RTS smoothing. It is a library: callers provide clean, aligned data and select or supply the reservoir configuration.
+
+## Package layout
+
+| Module | Responsibility |
+| --- | --- |
+| `kalmanflow.kalman` | Generic predict, update, and batch filtering primitives. |
+| `kalmanflow.rts` | Full-record and bounded online fixed-lag RTS smoothing. |
+| `kalmanflow.models` | The physical three-state reservoir model and model protocol. |
+| `kalmanflow.reservoir_backend` | Adapts the physical model to the streaming pipeline. |
+| `kalmanflow.pipeline` | Generic initialization, ordering, replay, and delayed-release coordinator. |
+| `kalmanflow.core` | Public batch and reservoir-streaming adapters. |
+| `kalmanflow.pandas_api` | DataFrame convenience wrapper for the default batch adapter. |
+| `kalmanflow.bayesian_tuning` | Public façade for Bayesian diagonal-noise search and compact frozen-configuration evaluation; implementation is split across preparation, diagnostics, candidate, optimization, proxy, and workflow modules. |
+| `kalmanflow.reservoir_config` | Immutable, validated per-reservoir configuration. |
+| `kalmanflow.observations` / `flags` | Public input and output-provenance types. |
+| `kalmanflow.units` | Volume/flow-rate conversion systems. |
+| `kalmanflow.time_utils` | UTC normalization and elapsed-time validation helpers. |
+
+The package-level imports are the supported starting point for applications.
+The generic Kalman, RTS, model, backend, and pipeline types are also exported
+for advanced integrations; their contracts are listed in the
+[API reference](API_REFERENCE.md). Checkpoint encoding and low-level validation
+modules are internal implementation details and are not public serialization
+formats or extension points.
+
+Repository-specific workflows live outside the package in `applications/`:
+`applications/preparation.py` parses and aligns the checked-in Aquarius
+exports, while `applications/run_bayesian_tuner.py` provides the offline
+Bayesian calibration entry point.
+`Notebooks/prepare_reservoir_data.py` remains a compatibility import for
+existing notebooks.
+
+## Reservoir state-space model
+
+The default state is `[storage, inflow_rate, true_outflow_rate]`. Storage and measured discharge are observations; discharge is not treated as a known control. For an interval `dt` seconds, the first state row is:
+
+```text
+storage(t + dt) = storage(t)
+                + dt × flow_to_volume_per_second
+                    × (inflow_rate(t) - true_outflow_rate(t))
+```
+
+The rate states are random walks. `ReservoirStateSpaceModel` derives the transition and discrete process covariance from the actual elapsed time and the continuous-time 3×3 diffusion covariance `q_continuous`.
+
+The `inflow_rate` state is the residual net balance contribution required by
+the supplied storage and accounted outflow. Measured outflow should include
+outlet releases, spills, and outward diversions or withdrawals as applicable.
+The API has no separate terms for precipitation, evaporation, seepage, or
+other water exchanges; applications must account for those separately with
+justified inputs or interpret them as model mismatch. Sensor bias, storage
+datum changes, and rating-curve changes are additional mismatch sources.
+Estimates are unconstrained by sign, so a negative value is mathematically
+valid even when a physical interpretation would prompt data or model review.
+
+`UnitSystem.us_customary()` is the default: storage is acre-feet and flow rates are cfs. `UnitSystem.si()` supports cubic metres and cubic metres per second; custom unit systems supply labels and a rate-to-volume-per-second conversion.
+
+## Public layers
+
+Use `get_reservoir_inflow` for a pandas batch result, or `OnlineReservoirInflow` for a reservoir-specific stream. The latter is built on the generic `OnlineInflowPipeline`, which can also be used with another backend and smoother implementation.
+
+The public reservoir layer exposes causal filtered inflow after its
+two-observation startup phase and later exposes an absolute fixed-lag RTS
+revised inflow for the same timestamp. The first finite-storage observation
+anchors the stream; it does not produce an inflow output by itself.
+Storage, measured outflow, and latent true outflow remain model inputs or
+internal state; no public outflow result is exposed. Generic filter and
+smoothing types remain available for advanced integrations.
+
+Offline calibration uses `tune_inflow_noise_bayesian` with clean, aligned
+storage and discharge series. It proposes a new immutable configuration but
+does not persist or activate it. `evaluate_configuration` scores one frozen
+configuration on a separate period without performing another search.
+
+## Data ownership and boundaries
+
+KalmanFlow does not read files, fetch or clean source data, align series, or
+deduplicate records. Callers manage reviewed configuration artifacts; the
+package validates the runtime contract—including timestamp indexes,
+matrix shapes, covariance properties, and missing observations—at the library
+boundary.

kalmanflow-0.1.0a1/Documentation/BAYESIAN_NOISE_TUNING.md

@@ -0,0 +1,217 @@
+# Experimental Bayesian innovation noise tuning
+
+Bayesian tuning is experimental. Its API, selection rules, and result schema
+may change between releases. Treat selected settings as proposals requiring
+independent validation before operational use.
+
+Install the optional search dependencies with `pip install "kalmanflow[tuning]"`.
+The optimizer loads only when `tune_inflow_noise_bayesian` is called. Importing
+KalmanFlow, filtering, smoothing, and `evaluate_configuration` work with the base
+installation. Existing package-level imports remain supported.
+
+## Scope and inputs
+
+`applications/run_bayesian_tuner.py` is the offline tuning workflow. Its
+`run_bayesian_tuner(...)` function accepts `project_root`, `reservoir`,
+`data_start`, `data_end`, `output_path`, and the optional
+`report_output_path`. It searches five diagonal covariance terms informed by
+the observed storage and outflow innovations:
+
+* `q_storage`, `q_inflow`, and `q_outflow`;
+* `r_storage` and `r_outflow`.
+
+The package API accepts already-cleaned `pandas.Series` inputs. Storage and
+discharge indexes must match exactly and must be timezone-aware, strictly
+increasing, unique, and nonmissing. The base configuration must use diagonal
+`q`, `r`, and `p0` matrices with positive finite values for all five searched
+diagonal `q`/`r` terms. At least three distinct positive
+`inflow_increment_sd_seeds` and three uniquely named, non-overlapping,
+half-open `ValidationWindow` intervals are required.
+Every supplied seed is evaluated. Consequently, `total_trials` must be at
+least the number of seeds; when `initial_trials` is smaller, the initial design
+is expanded to include them all.
+
+## Objective and selection
+
+The tuner scores the causal, pre-update predictive innovation distribution. It
+does not use true inflow values or smoothed states. Only rows inside the
+declared validation windows contribute to the objective, although earlier
+observations outside a window may causally condition its starting state. The
+supplied hourly inflow increment seeds are used as deterministic initial trials
+and define the `q_inflow` bounds.
+Remaining trials are selected by expected improvement from a Gaussian-process
+surrogate in normalized log-parameter space. Proxy diagnostics use only the
+same warmup-adjusted validation rows as the innovation objective.
+
+Each trial's objective is the validation-window-weighted joint predictive
+negative log density per observed component, plus the configured multiple of
+its between-window standard error. Hard validity checks exclude failed or
+insufficiently scored trials. Statistically competitive trials are identified
+with paired window differences; final selection prefers lower calibration
+violation, then proximity to the reviewed base configuration, then objective.
+
+`eligible` and `calibrated` answer different questions in both search and frozen
+evaluation. An eligible result has the required number of finite scored storage
+observations in every window and finite joint NLPD values. A search result is
+calibrated only when it is eligible and satisfies the configured NIS,
+innovation-bias, and autocorrelation checks. Frozen evaluation reports
+calibrated only when it is eligible and has no configured NIS or innovation-bias
+warnings; the search-only autocorrelation threshold is not applied by
+`evaluate_configuration`. Disabling calibration warning thresholds does not
+make an empty or numerically invalid evaluation eligible or calibrated.
+
+The elapsed-time autocorrelation diagnostic centers the finite innovations and
+normalizes each tolerance bin with the energy of the actual paired values. A
+bin with fewer than three finite pairs is reported as unavailable rather than
+being used as a calibration pass.
+
+## Output files and held-out evaluation
+
+The runner writes the proposed immutable `ReservoirConfig` to `output_path`.
+This compact configuration JSON is the default artifact and contains the
+complete configuration needed by a downstream application. Without an
+explicit `output_path`, it is written beneath `Outputs/bayesian_tuning`.
+
+Pass `report_output_path` to additionally write a detailed tuning audit. The
+report contains the compact candidate table (trial, acquisition source, five
+parameters, objective, eligibility, and selection flags), selected-trial
+per-window efficacy, compact upstream-proxy diagnostics when supplied, and a
+reference to the selected configuration artifact. It does not duplicate the
+full configuration JSON. The two resolved output paths must be different.
+
+A separate final evaluation interval should be reserved and evaluated with
+`evaluate_configuration` and an explicit `ValidationWindow` after tuning. The
+application runner divides its requested segment into three calibration
+windows; it does not reserve a final test segment automatically.
+
+Runtime diagnostics distinguish Gaussian-process `acquisition_seconds` from
+`candidate_evaluation_seconds`, which covers each compact causal evaluation.
+
+By default, the Bayesian search cannot increase `q_storage` or `r_storage`
+above the reviewed base configuration (`0.5x` to `1.0x`). This prevents an
+innovation-only objective from explaining storm-driven storage changes as
+extra storage process or measurement noise. These bounds remain configurable
+through `BayesianTuningSettings` when independent evidence supports a wider
+range. The inflow and outflow terms retain wider base-relative bounds.
+
+## Optional upstream proxy
+
+An upstream gauge may be supplied as an optional diagnostic proxy:
+
+```python
+from datetime import timedelta
+
+from kalmanflow import (
+    BayesianEvaluationSettings,
+    BayesianTuningSettings,
+    tune_inflow_noise_bayesian,
+)
+
+evaluation_settings = BayesianEvaluationSettings()
+
+result = tune_inflow_noise_bayesian(
+    storage,
+    discharge,
+    base_config,
+    inflow_increment_sd_seeds,
+    validation_windows,
+    upstream_proxy=upstream_flow,
+    settings=evaluation_settings,
+    bayesian_settings=BayesianTuningSettings(
+        proxy_max_lag=timedelta(hours=12),
+        proxy_diagnostic_frequency="1h",
+        proxy_min_shape_correlation=0.20,
+        proxy_min_change_correlation=0.10,
+    ),
+    proposed_configuration_version="reviewed-bayesian-v1",
+)
+```
+
+Proxy checks first match the gauge to the model timestamps, then perform a
+bounded lag search using standardized level and change shape. They use only
+the warmup-adjusted validation rows and aggregate at an hourly cadence by
+default. Only whole diagnostic-cadence shifts whose duration is within
+`proxy_max_lag` are considered, while zero lag remains available for a
+sub-cadence maximum. A positive reported lag means the proxy is shifted later
+relative to the causal estimate. They report best lag, level/change correlation,
+normalized shape RMSE, and a gate flag in the result and JSON report. Proxy magnitude is
+deliberately not matched and the proxy is never added to the Kalman observation
+vector or treated as true total inflow. If at least one innovation-valid trial
+passes the configured gate, selection is restricted to those trials; if none
+passes, the result is returned with an explicit diagnostic-only warning.
+
+The runtime preparation helper is `applications.preparation`. It retains
+`upstream_flow` and spillway audit columns in `PreparedReservoirData`. When the
+requested window contains at least one finite, nonnegative spillway value,
+finite spillway values are added to outlet discharge; missing or negative
+spillway values make combined outflow missing at those timestamps instead of
+being treated as zero. If the spillway file is absent or the requested window
+contains no usable spillway value, outlet-only outflow is preserved and the
+window audit records that limitation. The application tuner uses this helper;
+`Notebooks.prepare_reservoir_data` is retained only as a compatibility import
+for existing notebooks.
+
+## Application runner
+
+Run the workflow from the project root:
+
+```bash
+uv run --extra tuning python applications/run_bayesian_tuner.py
+```
+
+Direct execution uses the reviewed constants near the top of the runner;
+edit those constants first or call `run_bayesian_tuner(...)` from Python with
+explicit arguments. Without an explicit `output_path`, the JSON configuration is
+written beneath `Outputs/bayesian_tuning` as the compact selected
+configuration. Pass `report_output_path` to additionally write the detailed
+tuning audit report; it must resolve to a different file.
+
+For example, to produce both artifacts explicitly:
+
+```python
+from pathlib import Path
+
+from applications.run_bayesian_tuner import run_bayesian_tuner
+
+run_bayesian_tuner(
+    output_path=Path("Outputs/bayesian_tuning/chesbro-config.json"),
+    report_output_path=Path("Outputs/bayesian_tuning/chesbro-report.json"),
+)
+```
+
+The package API is:
+
+```python
+from kalmanflow import (
+    BayesianEvaluationSettings,
+    BayesianTuningSettings,
+    ValidationWindow,
+    evaluate_configuration,
+    tune_inflow_noise_bayesian,
+)
+
+result = tune_inflow_noise_bayesian(
+    storage,
+    discharge,
+    base_config,
+    inflow_increment_sd_seeds,
+    validation_windows,
+    settings=BayesianEvaluationSettings(),
+    bayesian_settings=BayesianTuningSettings(total_trials=40, initial_trials=12),
+    proposed_configuration_version="reviewed-bayesian-v1",
+)
+
+test_result = evaluate_configuration(
+    test_storage,
+    test_discharge,
+    result.selected_config,
+    evaluation_window=ValidationWindow("final-test", test_start, test_end),
+    settings=BayesianEvaluationSettings(),
+)
+```
+
+`evaluate_configuration` performs no search and does not modify the supplied
+configuration. It returns a compact `ConfigEvaluationResult` containing one
+configuration summary, per-window causal diagnostics, and warnings. When
+`evaluation_window` is omitted it evaluates the full supplied series, but an
+explicit untouched window is recommended for final approval.

kalmanflow-0.1.0a1/Documentation/CONFIGURATION.md

@@ -0,0 +1,90 @@
+# Configuration
+
+## Complete configuration example
+
+Use `ReservoirConfig` when a reservoir has reviewed noise settings, units, and metadata. The values below are structurally valid examples, not recommended production parameters.
+
+```python
+from datetime import timedelta
+
+import numpy as np
+
+from kalmanflow import (
+    InflowUnits,
+    InitializationStrategy,
+    ReservoirConfig,
+    UnitSystem,
+)
+
+config = ReservoirConfig(
+    reservoir_id="lexington",
+    reservoir_name="Lexington Reservoir",
+    q=np.diag([1.0, 0.01, 0.01]),
+    r=np.diag([100.0, 25.0]),
+    p0=np.diag([1_000.0, 100.0, 100.0]),
+    smoothing_lag=timedelta(hours=12),
+    initialization_strategy=InitializationStrategy.FIRST_TWO_VALID_STORAGE,
+    inflow_units=InflowUnits.CUBIC_FEET_PER_SECOND,
+    model_version="physical-rate-v1",
+    configuration_version="2026-01",
+    metadata={"source": "initial calibration"},
+    unit_system=UnitSystem.us_customary(),
+)
+```
+
+Then pass it to either adapter:
+
+```python
+from kalmanflow import OnlineReservoirInflow, get_reservoir_inflow_from_config
+
+stream = OnlineReservoirInflow.from_config(config)
+batch_result = get_reservoir_inflow_from_config(storage, discharge, config)
+```
+
+Both adapters use discharge as an observation input, but only return causal
+and revised inflow. A revised inflow is an absolute fixed-lag-smoothed value
+that replaces the causal inflow at its timestamp; it is not an adjustment to
+add. The latent true outflow state is never a public result.
+
+## Matrix and unit conventions
+
+The state order is `[storage, inflow_rate, true_outflow_rate]`.
+
+| Field | Shape | Meaning |
+| --- | --- | --- |
+| `q` | 3×3 | Continuous-time process-diffusion covariance; `q[i,j]` has units `state_i × state_j / second`. |
+| `r` | 2×2 | Measurement covariance for `[storage, measured_outflow]`; `r[i,j]` has units `observation_i × observation_j`. |
+| `p0` | 3×3 | Initial covariance; `p0[i,j]` has units `state_i × state_j`. |
+
+All covariance matrices must be finite, symmetric, and positive semidefinite. The two diagonal elements of `r` must be strictly positive. KalmanFlow derives each discrete process covariance from `q` and the actual elapsed time between observations, so never reuse a discrete, fixed-interval Q matrix as `q`.
+
+The balance model infers a net storage balance contribution. Measured outflow
+should include outlet releases, spills, and outward diversions or withdrawals
+as applicable. Precipitation, evaporation, seepage, and other water exchanges
+are not separate model terms; account for them with separate justified inputs
+or document them as model mismatch. Sensor bias, storage-datum changes, and
+rating-curve changes are additional mismatch sources. Estimates are not
+constrained to be nonnegative.
+
+`UnitSystem.us_customary()` uses acre-feet and cfs; `UnitSystem.si()` uses m³ and m³/s. A custom `UnitSystem` must supply a positive `flow_to_volume_per_second` conversion that matches the storage unit.
+
+## Selecting parameters
+
+Choose `q`, `r`, and `p0` through an engineering review of each reservoir's
+sensor accuracy, operating conditions, and historical data. Record the
+rationale in `metadata`, version the reviewed configuration, and validate it
+against a separate period before operational use.
+
+## Bayesian inflow-noise tuning
+
+For an offline proposal, use `tune_inflow_noise_bayesian` with a fixed
+`ReservoirConfig` and predeclared `ValidationWindow` intervals. The supplied
+`inflow_increment_sd_seeds` initialize the bounded Bayesian search for the five
+diagonal covariance terms. Bayesian evaluation requires `q`, `r`, and `p0` to
+be diagonal. The tuner runs a causal chronology per trial and scores the joint
+predictive innovation for whichever storage and outflow components are
+observed, before that row is assimilated. The returned configuration is
+proposed only; it is never persisted automatically. Supply a new
+`proposed_configuration_version`, review the compact report, and use
+`evaluate_configuration` with an explicit untouched `ValidationWindow` before
+operational approval.

kalmanflow-0.1.0a1/Documentation/INFLOW_MODEL_BEHAVIOR.md

@@ -0,0 +1,84 @@
+# KalmanFlow model behavior
+
+## Input contract
+
+`get_reservoir_inflow` and `get_reservoir_inflow_from_config` accept pre-cleaned storage and discharge `pandas.Series` with exactly equal, timezone-aware, strictly increasing indexes. Observation timestamps support microsecond precision; finer pandas timestamps are rejected before computation. The default adapter expects storage in acre-feet and discharge in cfs. The configured adapter uses the labels in `ReservoirConfig.unit_system`.
+
+`OnlineReservoirInflow.process` accepts either an `Observation` or `timestamp`, `storage`, and `discharge` keywords. Timestamps must be timezone-aware, no finer than microseconds, and strictly later than the previously accepted observation. All elapsed-time comparisons are normalized to UTC. Validation-window boundaries may retain nanosecond precision because they are interval labels rather than observations.
+
+The package intentionally does not sort, align, deduplicate, interpolate, or impute inputs.
+
+## Initialization and missing values
+
+The stream ignores leading rows with missing storage. Its first finite storage
+reading must also have finite discharge; that row becomes the initialization
+anchor. The next finite storage reading completes initialization, even if its
+discharge is missing. At the anchor timestamp, the model uses measured outflow
+as a steady-state inflow prior (zero initial storage-change assumption). That
+first filtered value therefore depends only on the anchor observation, not on
+the later storage reading. The second storage reading then updates inflow
+through the normal causal predict/update step. A finite storage row with
+missing discharge before an anchor is invalid rather than silently skipped.
+
+After initialization, either storage or discharge may be `NaN`. A finite component is still used as a partial Kalman observation. If both are `NaN`, the step is predict-only. Positive and negative infinity are invalid; `NaN` is the only missing-value marker. A public estimate carrying any missing observation component receives the `PREDICTED` flag; fully observed steps are `NORMAL`.
+
+## Batch outputs
+
+The batch adapters return a `DataFrame`, indexed like the inputs, with:
+
+| Column | Meaning |
+| --- | --- |
+| `estimated_inflow` | Causal filtered inflow rate. |
+| `revised_inflow` | Absolute fixed-lag-smoothed inflow replacement, or `NaN` until final. |
+| `estimated_inflow_flag` / `revised_inflow_flag` | `NORMAL` or `PREDICTED`. |
+| `estimated_inflow_smoothing_flag` | Always `NON_SMOOTHED`: inflow is causal. |
+| `revised_inflow_smoothing_flag` | `SMOOTHED` when released; otherwise `NON_SMOOTHED`. |
+
+When a revised inflow becomes available, it replaces `estimated_inflow` at the
+same timestamp; it is not a delta to add to that value. Storage and outflow
+are internal states and are not public batch outputs.
+
+## Streaming outputs
+
+Each call to `OnlineReservoirInflow.process` returns a `ReservoirFlowUpdate`.
+`filtered_inflows` contains newly available causal inflow estimates;
+`revised_inflows` contains only newly finalized, absolute smoothed
+replacements at their original timestamps. The initial successful call that
+completes initialization can emit two filtered inflow estimates. The active
+smoothing window is retained internally and is bounded by `max_window_steps`.
+Although both startup estimates are emitted together, the anchor estimate was
+computed from the anchor observation alone.
+
+The anchor state mean is seeded from the anchor storage and measured outflow,
+then that same anchor observation is assimilated by the filter. Its innovation
+is therefore zero by construction and the covariance can be reduced. This startup
+covariance is an engineering approximation rather than a measurement-
+conditioned uncertainty calibration; interpret early uncertainty and early
+calibration diagnostics accordingly.
+
+`process_many` is transactional: if any item is invalid, the stream returns to its entry state and produces no partial group result.
+
+## Configuration
+
+`ReservoirConfig` is immutable. It holds a reservoir identifier, model and configuration versions, continuous-time `q` (3×3), measurement covariance `r` (2×2), initial covariance `p0` (3×3), a positive smoothing lag, units, and metadata. Covariances must be finite, symmetric, positive semidefinite; the diagonal of `r` must be strictly positive.
+
+Select and review continuous-time process noise (`q`), observation noise (`r`),
+and initial covariance (`p0`) for each reservoir before operational use.
+Document the rationale and configuration version in the configuration metadata.
+
+## Bayesian causal evaluation
+
+`tune_inflow_noise_bayesian` is an offline calibration aid, not an adaptive
+streaming mode. It uses the forward Kalman filter and never calls the RTS
+smoother. Initialization rows and the configured warm-up period are excluded
+from scores. Only rows inside the declared validation windows contribute to
+the objective. Earlier observations outside a window may still condition its
+causal starting state, and validation observations can condition later
+predictions, as they would in operation; no observation can affect its own or
+an earlier score.
+
+Use `evaluate_configuration` once a proposed configuration is frozen for a
+separate, untouched test period. Test results must not be used to change the
+Bayesian search, windows, thresholds, or selected parameters. Filtered storage
+closure is a reconstruction diagnostic; it is not an independent predictive
+score when the ending storage observation has already been assimilated.

kalmanflow-0.1.0a1/Documentation/ONLINE_INFLOW_PIPELINE.md

@@ -0,0 +1,63 @@
+# Online inflow pipeline
+
+`OnlineInflowPipeline` coordinates an ordered observation stream with a model-specific backend and a fixed-lag smoother. `OnlineReservoirInflow` is the reservoir-facing wrapper and is the normal choice for application code.
+
+## Lifecycle
+
+1. The pipeline accepts timezone-aware, strictly increasing observations with at most microsecond timestamp precision. Nanosecond-level `ValidationWindow` boundaries remain valid for half-open scoring intervals.
+2. It ignores leading missing-storage rows, then stores the first finite-storage observation (which must have finite discharge) and waits for the next finite storage sample to initialize the backend.
+3. Initialization creates two forward filter steps. The first uses outflow as a steady-state inflow prior and does not inspect the second observation; the second is a normal causal update. Each later input creates one forward step, using the actual elapsed seconds since its predecessor.
+4. Each step enters the fixed-lag RTS smoother. A state is released only after the configured elapsed-time lag has passed.
+
+`PipelineUpdate.filtered_state` is the latest forward step; `filtered_states` contains every step created by that call; and `smoothed_states` contains just the newly finalized states. `provisional_states` may inspect the active lag window without releasing it.
+
+## Reservoir API
+
+Create `OnlineReservoirInflow` with scalar noise parameters for the default acre-ft/cfs model, or use `OnlineReservoirInflow.from_config(config)` for a validated configuration:
+
+```python
+from kalmanflow import Observation, OnlineReservoirInflow
+
+stream = OnlineReservoirInflow(
+    q_storage=1.0, q_inflow=1.0, q_outflow=1.0,
+    r_storage=1.0, r_outflow=1.0,
+)
+update = stream.process(Observation(timestamp, storage, discharge))
+```
+
+`filtered_inflows` are causal and marked `NON_SMOOTHED`. `revised_inflows`
+are released only after smoothing and marked `SMOOTHED`; each is an absolute
+inflow value that replaces the causal estimate at the same timestamp.
+Only `NaN` denotes a missing storage or discharge value; infinities are
+rejected before pipeline state changes.
+
+## Checkpoints
+
+Configured reservoir streams support compact checkpoints after a successful `process` or `process_many` call:
+
+```python
+checkpoint = stream.checkpoint()
+restored = OnlineReservoirInflow.from_checkpoint(checkpoint, config=config)
+```
+
+A checkpoint is bound to one `reservoir_id` and a fingerprint of the model,
+covariance, smoothing-lag, and unit settings. Restore rejects a configuration
+that does not match. Restoration rebuilds the active smoothing window from its
+retained observations without re-emitting already released records.
+
+Checkpoints are version-bound. Checkpoints created before the package rename
+are rejected during restore and must be recreated.
+
+Checkpoint timestamps preserve microsecond precision. Inputs with a nonzero
+nanosecond remainder are rejected before pipeline state changes, so restoring a
+checkpoint cannot silently change an accepted observation label.
+
+Checkpoints are unavailable during processing, before a successful `process` or
+`process_many` result, after a failed processing call, or for streams created
+without a reservoir ID. The serialized bytes are intentionally an internal
+format; retain the compatible configuration and use the package's restore API
+instead of decoding them yourself.
+
+## Resource bounds and failures
+
+`max_window_steps` must be at least two and bounds the active smoother window. Exceeding that bound, invalid timestamps, incompatible checkpoint data, or a backend failure raises an exception. `process_many` rolls back its entire input group on failure; single `process` validates before forwarding values to the pipeline.