anytimemonitor 0.9.0__tar.gz

anytimemonitor-0.9.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jonathan Chevelu and Nelly Barbot (IRISA, Université de Rennes)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

anytimemonitor-0.9.0/PKG-INFO

@@ -0,0 +1,269 @@
+Metadata-Version: 2.4
+Name: anytimemonitor
+Version: 0.9.0
+Summary: Adaptive listening-test controller for TTS-MOS evaluation: anytime-valid confidence sequences, per-pair verdicts, two-lever (items/listeners) budget advice
+Author: Jonathan Chevelu, Nelly Barbot
+License: MIT
+Project-URL: Homepage, https://expression.gitlabpages.inria.fr/replication/2026-doing-mos-well/
+Project-URL: Source, https://github.com/jchevelu/AnytimeMonitor
+Project-URL: Repository, https://gitlab.inria.fr/expression/tools/anytimemonitor
+Project-URL: Archive, https://doi.org/10.5281/zenodo.21326813
+Keywords: speech-synthesis,MOS,listening-test,anytime-valid,confidence-sequence,evaluation
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.26
+Requires-Dist: pandas>=2.1
+Requires-Dist: scipy>=1.11
+Requires-Dist: matplotlib>=3.8
+Provides-Extra: ui
+Requires-Dist: streamlit>=1.50; extra == "ui"
+Provides-Extra: test
+Requires-Dist: pytest>=7; extra == "test"
+Dynamic: license-file
+
+# AnytimeMonitor — adaptive controller for TTS-MOS listening tests
+
+AnytimeMonitor watches a MOS listening test *while it runs* and tells
+the test admin, for every pair of systems, whether the data already
+supports a verdict — and if not, which lever (more items, or more
+listeners) closes the pair fastest.
+
+The statistical core is an **anytime-valid confidence sequence**
+(Howard, Ramdas, McAuliffe & Sekhon, 2021) on per-item paired score
+differences. Anytime validity means the coverage guarantee holds at
+*any* data-dependent stopping time: the admin can look at the verdicts
+as often as they like, stop whenever a pair resolves, and the reported
+confidence interval is still valid. No peeking penalty bookkeeping, no
+pre-registered sample size required.
+
+This is a research prototype (v0.9). It accompanies a position paper
+on budget allocation in TTS evaluation (under submission, *Computer
+Speech & Language* Special Issue on the evaluation of speech and
+speech synthesis).
+
+## What the admin sees
+
+For each pair of systems (A, B), one of six verdicts:
+
+| Verdict | Meaning |
+| --- | --- |
+| `STOP` | The anytime-valid CI excludes 0 (0 ∉ CI): the pair is separated. |
+| `STOP_EQUIV` | Strict TOST equivalence: CI ⊂ ±δ_equiv *and* the half-width Ψ_eff ≤ δ_equiv/2. The pair is statistically equivalent within tolerance. |
+| `CONTINUE +N` | Neither verdict yet. The tool predicts how many more **items** (+N) or **listeners** (+L) would close the pair, and which terminal (STOP or STOP_EQUIV) it is heading for. The headline is the cheaper option. |
+| `WAIT` | The listener×system variance component cannot be estimated yet (fewer than 3 listeners with ≥ 2 paired items). Structural: keep collecting before trusting any verdict. |
+| `CORPUS_LIMIT` | The item budget is exhausted *and* the listener lever can no longer close the pair. Terminal. |
+| `LISTENER_LIMIT` | The listener budget is exhausted *and* the item lever can no longer close the pair. Terminal. |
+
+> **Verdict naming.** `STOP` is the separation verdict the companion paper
+> calls `STOP_SEP`. `STOP` is the current public label and will be **renamed to
+> `STOP_SEP` in a future release** (to pair symmetrically with `STOP_EQUIV`);
+> code matching on `"STOP"` should expect that change.
+
+Verdicts are recomputed from scratch at every call — the controller is
+queried on demand (a handful of times over a test lifecycle), not in a
+real-time loop. A `STOP` is **not sticky**: if you keep collecting
+past it, a borderline pair can revert to `CONTINUE`. This is honest
+anytime-CI behaviour (the guarantee is on coverage, not on the
+monotonicity of the observed difference); in practice the admin acts
+on the first `STOP`.
+
+## Install
+
+```bash
+pip install -e .          # from the repository root (pyproject.toml)
+pip install -e ".[ui]"    # + Streamlit demo UI
+pip install -e ".[test]"  # + pytest; then run `pytest`
+```
+
+Requires Python ≥ 3.10. Pinned versions used for development are in
+`requirements.txt`.
+
+## Quick start
+
+### Programmatic
+
+```python
+import pandas as pd
+from anytimemonitor import AnytimeMonitor
+
+ctrl = AnytimeMonitor(mde_threshold=0.30)   # δ_sep, in MOS units
+
+# Feed ratings as they arrive — long format, one row per judgment
+ctrl.add_ratings(pd.DataFrame({
+    "listener_id": ...,   # who rated
+    "item":        ...,   # which sentence / stimulus
+    "system":      ...,   # which TTS system
+    "rating":      ...,   # the MOS score
+}))
+
+print(ctrl.report())                  # all pairs
+dec = ctrl.decision("sysA", "sysB")   # one pair, structured
+print(dec.verdict, dec.ci_lo, dec.ci_hi, dec.n_more_items)
+```
+
+### CLI demo
+
+Feed a ratings CSV to the controller batch by batch and watch the per-pair
+verdicts tighten. With no arguments it runs on a bundled synthetic sample:
+
+```bash
+python -m anytimemonitor.demo                          # bundled sample
+python -m anytimemonitor.demo --input my_ratings.csv   # your own long-format CSV
+```
+
+### Streamlit demo UI
+
+```bash
+streamlit run anytimemonitor/ui/app.py
+```
+
+Upload a ratings CSV (or pick a bundled synthetic example from
+`sample_uploads/`), watch the controller replay the stream
+batch-by-batch, and drag the time-machine slider to see the verdict
+and two-lever advice at any past timestep. See `ui/README.md`.
+
+## How it works
+
+For a pair (A, B), the controller builds per-item paired differences
+d_j = mean rating of item j under A minus under B, and maintains a
+sub-Gaussian normal-mixture confidence sequence on their mean
+(Howard et al. 2021). Two ingredients adapt this to listening-test
+data, where ratings are clustered by listener as well as by item:
+
+- **Bivariate cluster bootstrap SE.** Resampling *items* gives the
+  i.i.d.-part SE that the anytime boundary multiplies; resampling
+  *listeners* gives the listener-pool floor ("plancher") — the
+  precision ceiling that no amount of extra items can break, added in
+  quadrature. When the data is too sparse to bootstrap, the
+  controller falls back to the confidence sequence's internal running
+  variance proxy.
+- **Variance decomposition layer.** Method-of-moments estimates of
+  the system×item and residual components (per system, with a
+  Half-Cauchy Bayesian regulariser that prevents the
+  truncation-at-zero bias of raw MoM), plus a per-pair estimate of
+  the listener×system component. These feed the two-lever
+  prediction: +N items shrinks the interaction term, +L listeners
+  lowers the floor.
+
+The price of anytime validity is a modestly wider interval than a
+fixed-n test (a √log log factor), and deliberate conservatism early
+on: below `avci_t_min_ebernstein` items the variance proxy falls back
+to the worst-case Hoeffding bound, so the first batches produce wide
+intervals. The controller is therefore most useful from a few dozen
+rated items per system onward.
+
+The internal architecture is documented in
+[`Architecture.md`](Architecture.md) (and
+[`ui/Architecture.md`](ui/Architecture.md) for the demo UI); the deeper design
+notes for the confidence sequence are in [`AVCI_PLAN.md`](AVCI_PLAN.md).
+
+## Key parameters
+
+All knobs are constructor arguments of `AnytimeMonitor`; the
+defaults are deliberately conservative.
+
+| Parameter | Default | Role |
+| --- | --- | --- |
+| `mde_threshold` (δ_sep) | (required) | The smallest effect worth *resolving* (MOS): seeds `delta_equiv` and the reported MDE. It does **not** gate `STOP` — separation fires on `0 ∉ CI` regardless. |
+| `delta_equiv` | `None` (= `mde_threshold`) | δ_equiv — equivalence margin for `STOP_EQUIV`. Set it below `mde_threshold` to create an indifference zone. |
+| `alpha` | 0.05 | Nominal level of the confidence sequence. |
+| `power` | 0.80 | Only sets the reported-MDE scale factor (`z_{α/2} + z_power`); does not affect stopping. |
+| `n_max_corpus`, `l_max_listeners` | `None` | Item / listener budgets; enable the `*_LIMIT` terminal verdicts. |
+| `avci_n_planned` | `240` (§7 census median budget) | Planned item budget; the controller derives η = 1/√(N_planned) (Howard's a-priori rule). |
+| `avci_eta` | `None` (→ 1/√`avci_n_planned`) | Set the Howard 2021 mixing parameter η by hand instead of via `avci_n_planned` — η is fixed a priori, never tuned to data. **Pass one or the other, not both** (raises otherwise). |
+| `avci_lissys_floor_r0` | `0.36` (measured r_lisSys band frontier) | A-priori guard on the listener floor: bounds it below by √(2·σ²₀/L) with σ²₀ = 1.5·r₀·σ̂²_eval (the ×1.5 is an MC-calibrated validity margin). Protects coverage when a small pool under-estimates σ²_lisSys. `0` disables. |
+| `avci_lissys_floor_sigma2` | `None` | Absolute σ²₀ override for the guard (expert path, no extra margin). **Exclusive with `avci_lissys_floor_r0`** (raises otherwise). The per-pair `Decision.lissys_prior_exceeded` flags when data-driven floor estimates exceed the declared guard (declared margin exhausted; consider raising r₀). |
+| `avci_gamma_alpha_factor` | 1.0 | Multiplier `c` on the listener floor (√c on the plancher); `1.0` assumes σ²_lisSys ≤ σ²_int. Bypassed when the bootstrap plancher is available (the normal path). |
+| `avci_proxy` | `"ebernstein"` | Variance proxy: running empirical variance (`ebernstein`) or worst-case `hoeffding`. |
+| `avci_d_bounds` | `(-4, 4)` | Range of the per-item paired *difference* (not the raw rating scale): ±(rating max − min), so the default spans a rating range of 4. The normal path bootstraps the SE from the data and **ignores this** (it feeds only the sparse-data fallback and a reported diagnostic); widen it for a larger rating range if you drive `AnytimeCI` directly on sparse data. |
+| `avci_t_min_ebernstein` | 10 | Below this item count the `ebernstein` proxy falls back to `hoeffding` (fallback path only). |
+| `avci_ebernstein_floor_frac` | 0.01 | Relative floor on the empirical-variance proxy, as a fraction of the `hoeffding` bound (fallback path only). |
+| `avci_bootstrap_B` | 200 | Bootstrap resamples per decision call. |
+| `avci_bootstrap_seed` | 0 | PRNG seed for the bivariate cluster bootstrap. |
+| `sigma2_ga_aggregator` | `"median"` | Which bootstrap-CI point anchors the listener floor: `median` / `mean` / `q95` (conservative) / `None`. |
+| `cold_start_ga_ratio` | 2.7 | Conservative BTTF-anchored prior ratio (σ²_lisSys/σ²_int) used before the component is identifiable. |
+| `avci_wait_min_listeners_kge2` | 3 | Structural WAIT gate: minimum listeners with ≥ 2 paired items. |
+| `avci_wait_max_ci_ratio` | 5.0 | WAIT gate: max tolerated q95/median on the σ²_lisSys bootstrap CI before abstaining. |
+
+> **Legacy naming.** Parameters and `Decision` fields carrying `ga` /
+> `gamma_alpha` (e.g. `cold_start_ga_ratio`, `sigma2_ga_*`) name the
+> listener×system variance the paper writes `σ²_lisSys`; the identifiers will be
+> **renamed to `lissys` in a future release** (an API change, hence deferred).
+
+## Input data requirements
+
+- Long format, columns `listener_id`, `item`, `system`, `rating`
+  (extra columns are ignored).
+- Ratings on any bounded scale. The controller works on per-item paired
+  *differences*, whose extremes are ±(rating max − min); the default
+  `avci_d_bounds = (-4, 4)` spans a rating range of 4. On the normal path the
+  standard error is bootstrapped from the data, so the scale is handled
+  automatically and this default is safe on any scale; `avci_d_bounds` only
+  enters the sparse-data fallback (when the cluster bootstrap cannot run).
+- The design must let listeners rate **multiple items per system**
+  (K ≥ 2 somewhere): a design where each listener rates a single item
+  — or each item is rated by a single listener — cannot identify the
+  listener×system component, and the controller will honestly report
+  `WAIT` forever rather than emit verdicts blind to that term.
+
+## Repository layout
+
+```text
+anytimemonitor/
+├── __init__.py           ← public API: AnytimeMonitor, Decision,
+│                           VarianceEstimate, estimate_components
+├── controller.py         ← the AnytimeMonitor controller
+├── avci.py               ← AnytimeCI — Howard 2021 sub-Gaussian
+│                           normal-mixture confidence sequence
+├── variance.py           ← MoM variance components + Half-Cauchy
+│                           regulariser
+├── demo.py               ← CLI example (batch-fed on a bundled sample)
+├── ui/                   ← Streamlit demo (app.py; see ui/README.md + ui/Architecture.md)
+├── sample_uploads/       ← synthetic example CSVs for the UI
+├── Architecture.md       ← internal architecture (controller + statistics)
+├── AVCI_PLAN.md          ← deeper design notes for the confidence sequence
+├── TODO.md               ← development roadmap
+├── archive/              ← retired code, notes and frozen runs
+├── LICENSE               ← MIT
+└── requirements.txt      ← pinned development stack
+```
+
+The validation and simulation harnesses (synthetic-data generator,
+controller-vs-oracle simulator, false-positive-rate check) are not part
+of the installed package; they ship with the companion paper's
+reproduction code.
+
+## Status and caveats
+
+- **Research prototype.** The API may still change. A `pytest` smoke-test
+  suite (`anytimemonitor/tests/`) checks the package runs end to end; the
+  controller has additionally been validated by simulation
+  (coverage under H₀, controller-vs-oracle gap decomposition) and
+  exercised on Blizzard Challenge data; it has not yet been plugged
+  into a live test platform.
+- **Planning aid first.** The anytime-valid CI is a valid inference
+  under optional stopping, so it can be reported as such — but the
+  two-lever predictions (+N items / +L listeners) are noisy
+  projections from running variance estimates, not guarantees.
+- A full-scale validation campaign (multi-seed, multi-corpus
+  false-positive and power sweeps) is planned alongside the companion
+  paper; the numbers shipped here come from smaller smoke-scale runs.
+
+## License
+
+MIT — see [LICENSE](LICENSE).
+
+## Citation
+
+A companion paper is under submission. Until it is out, please cite
+this repository directly.

anytimemonitor-0.9.0/README.md

@@ -0,0 +1,236 @@
+# AnytimeMonitor — adaptive controller for TTS-MOS listening tests
+
+AnytimeMonitor watches a MOS listening test *while it runs* and tells
+the test admin, for every pair of systems, whether the data already
+supports a verdict — and if not, which lever (more items, or more
+listeners) closes the pair fastest.
+
+The statistical core is an **anytime-valid confidence sequence**
+(Howard, Ramdas, McAuliffe & Sekhon, 2021) on per-item paired score
+differences. Anytime validity means the coverage guarantee holds at
+*any* data-dependent stopping time: the admin can look at the verdicts
+as often as they like, stop whenever a pair resolves, and the reported
+confidence interval is still valid. No peeking penalty bookkeeping, no
+pre-registered sample size required.
+
+This is a research prototype (v0.9). It accompanies a position paper
+on budget allocation in TTS evaluation (under submission, *Computer
+Speech & Language* Special Issue on the evaluation of speech and
+speech synthesis).
+
+## What the admin sees
+
+For each pair of systems (A, B), one of six verdicts:
+
+| Verdict | Meaning |
+| --- | --- |
+| `STOP` | The anytime-valid CI excludes 0 (0 ∉ CI): the pair is separated. |
+| `STOP_EQUIV` | Strict TOST equivalence: CI ⊂ ±δ_equiv *and* the half-width Ψ_eff ≤ δ_equiv/2. The pair is statistically equivalent within tolerance. |
+| `CONTINUE +N` | Neither verdict yet. The tool predicts how many more **items** (+N) or **listeners** (+L) would close the pair, and which terminal (STOP or STOP_EQUIV) it is heading for. The headline is the cheaper option. |
+| `WAIT` | The listener×system variance component cannot be estimated yet (fewer than 3 listeners with ≥ 2 paired items). Structural: keep collecting before trusting any verdict. |
+| `CORPUS_LIMIT` | The item budget is exhausted *and* the listener lever can no longer close the pair. Terminal. |
+| `LISTENER_LIMIT` | The listener budget is exhausted *and* the item lever can no longer close the pair. Terminal. |
+
+> **Verdict naming.** `STOP` is the separation verdict the companion paper
+> calls `STOP_SEP`. `STOP` is the current public label and will be **renamed to
+> `STOP_SEP` in a future release** (to pair symmetrically with `STOP_EQUIV`);
+> code matching on `"STOP"` should expect that change.
+
+Verdicts are recomputed from scratch at every call — the controller is
+queried on demand (a handful of times over a test lifecycle), not in a
+real-time loop. A `STOP` is **not sticky**: if you keep collecting
+past it, a borderline pair can revert to `CONTINUE`. This is honest
+anytime-CI behaviour (the guarantee is on coverage, not on the
+monotonicity of the observed difference); in practice the admin acts
+on the first `STOP`.
+
+## Install
+
+```bash
+pip install -e .          # from the repository root (pyproject.toml)
+pip install -e ".[ui]"    # + Streamlit demo UI
+pip install -e ".[test]"  # + pytest; then run `pytest`
+```
+
+Requires Python ≥ 3.10. Pinned versions used for development are in
+`requirements.txt`.
+
+## Quick start
+
+### Programmatic
+
+```python
+import pandas as pd
+from anytimemonitor import AnytimeMonitor
+
+ctrl = AnytimeMonitor(mde_threshold=0.30)   # δ_sep, in MOS units
+
+# Feed ratings as they arrive — long format, one row per judgment
+ctrl.add_ratings(pd.DataFrame({
+    "listener_id": ...,   # who rated
+    "item":        ...,   # which sentence / stimulus
+    "system":      ...,   # which TTS system
+    "rating":      ...,   # the MOS score
+}))
+
+print(ctrl.report())                  # all pairs
+dec = ctrl.decision("sysA", "sysB")   # one pair, structured
+print(dec.verdict, dec.ci_lo, dec.ci_hi, dec.n_more_items)
+```
+
+### CLI demo
+
+Feed a ratings CSV to the controller batch by batch and watch the per-pair
+verdicts tighten. With no arguments it runs on a bundled synthetic sample:
+
+```bash
+python -m anytimemonitor.demo                          # bundled sample
+python -m anytimemonitor.demo --input my_ratings.csv   # your own long-format CSV
+```
+
+### Streamlit demo UI
+
+```bash
+streamlit run anytimemonitor/ui/app.py
+```
+
+Upload a ratings CSV (or pick a bundled synthetic example from
+`sample_uploads/`), watch the controller replay the stream
+batch-by-batch, and drag the time-machine slider to see the verdict
+and two-lever advice at any past timestep. See `ui/README.md`.
+
+## How it works
+
+For a pair (A, B), the controller builds per-item paired differences
+d_j = mean rating of item j under A minus under B, and maintains a
+sub-Gaussian normal-mixture confidence sequence on their mean
+(Howard et al. 2021). Two ingredients adapt this to listening-test
+data, where ratings are clustered by listener as well as by item:
+
+- **Bivariate cluster bootstrap SE.** Resampling *items* gives the
+  i.i.d.-part SE that the anytime boundary multiplies; resampling
+  *listeners* gives the listener-pool floor ("plancher") — the
+  precision ceiling that no amount of extra items can break, added in
+  quadrature. When the data is too sparse to bootstrap, the
+  controller falls back to the confidence sequence's internal running
+  variance proxy.
+- **Variance decomposition layer.** Method-of-moments estimates of
+  the system×item and residual components (per system, with a
+  Half-Cauchy Bayesian regulariser that prevents the
+  truncation-at-zero bias of raw MoM), plus a per-pair estimate of
+  the listener×system component. These feed the two-lever
+  prediction: +N items shrinks the interaction term, +L listeners
+  lowers the floor.
+
+The price of anytime validity is a modestly wider interval than a
+fixed-n test (a √log log factor), and deliberate conservatism early
+on: below `avci_t_min_ebernstein` items the variance proxy falls back
+to the worst-case Hoeffding bound, so the first batches produce wide
+intervals. The controller is therefore most useful from a few dozen
+rated items per system onward.
+
+The internal architecture is documented in
+[`Architecture.md`](Architecture.md) (and
+[`ui/Architecture.md`](ui/Architecture.md) for the demo UI); the deeper design
+notes for the confidence sequence are in [`AVCI_PLAN.md`](AVCI_PLAN.md).
+
+## Key parameters
+
+All knobs are constructor arguments of `AnytimeMonitor`; the
+defaults are deliberately conservative.
+
+| Parameter | Default | Role |
+| --- | --- | --- |
+| `mde_threshold` (δ_sep) | (required) | The smallest effect worth *resolving* (MOS): seeds `delta_equiv` and the reported MDE. It does **not** gate `STOP` — separation fires on `0 ∉ CI` regardless. |
+| `delta_equiv` | `None` (= `mde_threshold`) | δ_equiv — equivalence margin for `STOP_EQUIV`. Set it below `mde_threshold` to create an indifference zone. |
+| `alpha` | 0.05 | Nominal level of the confidence sequence. |
+| `power` | 0.80 | Only sets the reported-MDE scale factor (`z_{α/2} + z_power`); does not affect stopping. |
+| `n_max_corpus`, `l_max_listeners` | `None` | Item / listener budgets; enable the `*_LIMIT` terminal verdicts. |
+| `avci_n_planned` | `240` (§7 census median budget) | Planned item budget; the controller derives η = 1/√(N_planned) (Howard's a-priori rule). |
+| `avci_eta` | `None` (→ 1/√`avci_n_planned`) | Set the Howard 2021 mixing parameter η by hand instead of via `avci_n_planned` — η is fixed a priori, never tuned to data. **Pass one or the other, not both** (raises otherwise). |
+| `avci_lissys_floor_r0` | `0.36` (measured r_lisSys band frontier) | A-priori guard on the listener floor: bounds it below by √(2·σ²₀/L) with σ²₀ = 1.5·r₀·σ̂²_eval (the ×1.5 is an MC-calibrated validity margin). Protects coverage when a small pool under-estimates σ²_lisSys. `0` disables. |
+| `avci_lissys_floor_sigma2` | `None` | Absolute σ²₀ override for the guard (expert path, no extra margin). **Exclusive with `avci_lissys_floor_r0`** (raises otherwise). The per-pair `Decision.lissys_prior_exceeded` flags when data-driven floor estimates exceed the declared guard (declared margin exhausted; consider raising r₀). |
+| `avci_gamma_alpha_factor` | 1.0 | Multiplier `c` on the listener floor (√c on the plancher); `1.0` assumes σ²_lisSys ≤ σ²_int. Bypassed when the bootstrap plancher is available (the normal path). |
+| `avci_proxy` | `"ebernstein"` | Variance proxy: running empirical variance (`ebernstein`) or worst-case `hoeffding`. |
+| `avci_d_bounds` | `(-4, 4)` | Range of the per-item paired *difference* (not the raw rating scale): ±(rating max − min), so the default spans a rating range of 4. The normal path bootstraps the SE from the data and **ignores this** (it feeds only the sparse-data fallback and a reported diagnostic); widen it for a larger rating range if you drive `AnytimeCI` directly on sparse data. |
+| `avci_t_min_ebernstein` | 10 | Below this item count the `ebernstein` proxy falls back to `hoeffding` (fallback path only). |
+| `avci_ebernstein_floor_frac` | 0.01 | Relative floor on the empirical-variance proxy, as a fraction of the `hoeffding` bound (fallback path only). |
+| `avci_bootstrap_B` | 200 | Bootstrap resamples per decision call. |
+| `avci_bootstrap_seed` | 0 | PRNG seed for the bivariate cluster bootstrap. |
+| `sigma2_ga_aggregator` | `"median"` | Which bootstrap-CI point anchors the listener floor: `median` / `mean` / `q95` (conservative) / `None`. |
+| `cold_start_ga_ratio` | 2.7 | Conservative BTTF-anchored prior ratio (σ²_lisSys/σ²_int) used before the component is identifiable. |
+| `avci_wait_min_listeners_kge2` | 3 | Structural WAIT gate: minimum listeners with ≥ 2 paired items. |
+| `avci_wait_max_ci_ratio` | 5.0 | WAIT gate: max tolerated q95/median on the σ²_lisSys bootstrap CI before abstaining. |
+
+> **Legacy naming.** Parameters and `Decision` fields carrying `ga` /
+> `gamma_alpha` (e.g. `cold_start_ga_ratio`, `sigma2_ga_*`) name the
+> listener×system variance the paper writes `σ²_lisSys`; the identifiers will be
+> **renamed to `lissys` in a future release** (an API change, hence deferred).
+
+## Input data requirements
+
+- Long format, columns `listener_id`, `item`, `system`, `rating`
+  (extra columns are ignored).
+- Ratings on any bounded scale. The controller works on per-item paired
+  *differences*, whose extremes are ±(rating max − min); the default
+  `avci_d_bounds = (-4, 4)` spans a rating range of 4. On the normal path the
+  standard error is bootstrapped from the data, so the scale is handled
+  automatically and this default is safe on any scale; `avci_d_bounds` only
+  enters the sparse-data fallback (when the cluster bootstrap cannot run).
+- The design must let listeners rate **multiple items per system**
+  (K ≥ 2 somewhere): a design where each listener rates a single item
+  — or each item is rated by a single listener — cannot identify the
+  listener×system component, and the controller will honestly report
+  `WAIT` forever rather than emit verdicts blind to that term.
+
+## Repository layout
+
+```text
+anytimemonitor/
+├── __init__.py           ← public API: AnytimeMonitor, Decision,
+│                           VarianceEstimate, estimate_components
+├── controller.py         ← the AnytimeMonitor controller
+├── avci.py               ← AnytimeCI — Howard 2021 sub-Gaussian
+│                           normal-mixture confidence sequence
+├── variance.py           ← MoM variance components + Half-Cauchy
+│                           regulariser
+├── demo.py               ← CLI example (batch-fed on a bundled sample)
+├── ui/                   ← Streamlit demo (app.py; see ui/README.md + ui/Architecture.md)
+├── sample_uploads/       ← synthetic example CSVs for the UI
+├── Architecture.md       ← internal architecture (controller + statistics)
+├── AVCI_PLAN.md          ← deeper design notes for the confidence sequence
+├── TODO.md               ← development roadmap
+├── archive/              ← retired code, notes and frozen runs
+├── LICENSE               ← MIT
+└── requirements.txt      ← pinned development stack
+```
+
+The validation and simulation harnesses (synthetic-data generator,
+controller-vs-oracle simulator, false-positive-rate check) are not part
+of the installed package; they ship with the companion paper's
+reproduction code.
+
+## Status and caveats
+
+- **Research prototype.** The API may still change. A `pytest` smoke-test
+  suite (`anytimemonitor/tests/`) checks the package runs end to end; the
+  controller has additionally been validated by simulation
+  (coverage under H₀, controller-vs-oracle gap decomposition) and
+  exercised on Blizzard Challenge data; it has not yet been plugged
+  into a live test platform.
+- **Planning aid first.** The anytime-valid CI is a valid inference
+  under optional stopping, so it can be reported as such — but the
+  two-lever predictions (+N items / +L listeners) are noisy
+  projections from running variance estimates, not guarantees.
+- A full-scale validation campaign (multi-seed, multi-corpus
+  false-positive and power sweeps) is planned alongside the companion
+  paper; the numbers shipped here come from smaller smoke-scale runs.
+
+## License
+
+MIT — see [LICENSE](LICENSE).
+
+## Citation
+
+A companion paper is under submission. Until it is out, please cite
+this repository directly.

anytimemonitor-0.9.0/anytimemonitor/__init__.py

@@ -0,0 +1,21 @@
+"""anytimemonitor — adaptive controller for TTS-MOS listening tests.
+
+Public API:
+    AnytimeMonitor — stateful controller built on an anytime-valid
+                        confidence sequence (per-item paired diffs).
+                        Add ratings, query for per-pair verdicts and
+                        two-lever (+items / +listeners) advice at any
+                        data-dependent time.
+    Decision          — dataclass returned by AnytimeMonitor.decision().
+
+See README.md for use case and architecture.
+"""
+from .controller import Decision, AnytimeMonitor
+from .variance import VarianceEstimate, estimate_components
+
+__all__ = [
+    "AnytimeMonitor",
+    "Decision",
+    "VarianceEstimate",
+    "estimate_components",
+]