ssbc 0.1.0__py3-none-any.whl

ssbc-0.1.0.dist-info/METADATA

@@ -0,0 +1,266 @@
+Metadata-Version: 2.4
+Name: ssbc
+Version: 0.1.0
+Summary: Small Sample Beta Correction - PAC guarantees with small datasets
+Author-email: Petrus H Zwart <phzwart@lbl.gov>
+Maintainer-email: Petrus H Zwart <phzwart@lbl.gov>
+License-Expression: MIT
+Project-URL: bugs, https://github.com/phzwart/ssbc/issues
+Project-URL: changelog, https://github.com/phzwart/ssbc/blob/master/changelog.md
+Project-URL: homepage, https://github.com/phzwart/ssbc
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Science/Research
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering
+Classifier: Topic :: Scientific/Engineering :: Mathematics
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: matplotlib
+Requires-Dist: numpy
+Requires-Dist: pandas
+Requires-Dist: plotly
+Requires-Dist: rich
+Requires-Dist: scipy
+Requires-Dist: typer
+Provides-Extra: test
+Requires-Dist: coverage; extra == "test"
+Requires-Dist: pytest; extra == "test"
+Requires-Dist: pytest-cov; extra == "test"
+Requires-Dist: ruff; extra == "test"
+Requires-Dist: ty; extra == "test"
+Requires-Dist: ipdb; extra == "test"
+Provides-Extra: dev
+Requires-Dist: pre-commit; extra == "dev"
+Requires-Dist: bandit[toml]; extra == "dev"
+Dynamic: license-file
+
+# SSBC: Small-Sample Beta Correction
+
+![PyPI version](https://img.shields.io/pypi/v/ssbc.svg)
+[![Documentation Status](https://readthedocs.org/projects/ssbc/badge/?version=latest)](https://ssbc.readthedocs.io/en/latest/?version=latest)
+
+**Small-Sample Beta Correction** provides PAC (Probably Approximately Correct) guarantees for conformal prediction with small calibration sets.
+
+* PyPI package: https://pypi.org/project/ssbc/
+* Free software: MIT License
+* Documentation: https://ssbc.readthedocs.io.
+
+## Overview
+
+SSBC addresses the challenge of constructing valid prediction sets when you have limited calibration data. Traditional conformal prediction assumes large calibration sets, but in practice, data is often scarce. SSBC provides finite-sample correction with PAC guarantees.
+
+### Key Features
+
+- ✅ **Small-Sample Correction**: PAC-valid conformal prediction for small calibration sets
+- ✅ **Mondrian Conformal Prediction**: Per-class calibration for handling class imbalance
+- ✅ **Comprehensive Statistics**: Detailed reporting with Clopper-Pearson confidence intervals
+- ✅ **Hyperparameter Tuning**: Interactive parallel coordinates visualization for parameter optimization
+- ✅ **Simulation Tools**: Built-in data generators for testing and validation
+
+## Installation
+
+```bash
+pip install ssbc
+```
+
+Or from source:
+
+```bash
+git clone https://github.com/yourusername/ssbc.git
+cd ssbc
+pip install -e .
+```
+
+## Quick Start
+
+```python
+import numpy as np
+from ssbc import (
+    ssbc_correct,
+    BinaryClassifierSimulator,
+    split_by_class,
+    mondrian_conformal_calibrate,
+    report_prediction_stats,
+)
+
+# 1. Generate simulated data
+sim = BinaryClassifierSimulator(
+    p_class1=0.1,
+    beta_params_class0=(2, 8),
+    beta_params_class1=(8, 2),
+    seed=42
+)
+labels, probs = sim.generate(n_samples=100)
+
+# 2. Split by class for Mondrian CP
+class_data = split_by_class(labels, probs)
+
+# 3. Calibrate with SSBC correction
+cal_result, pred_stats = mondrian_conformal_calibrate(
+    class_data=class_data,
+    alpha_target=0.10,  # 10% miscoverage
+    delta=0.10,         # 90% PAC guarantee
+    mode="beta"
+)
+
+# 4. Generate comprehensive report
+summary = report_prediction_stats(pred_stats, cal_result, verbose=True)
+```
+
+## Core Algorithm: SSBC
+
+The SSBC algorithm finds the optimal corrected miscoverage rate α' that satisfies:
+
+**P(Coverage(α') ≥ 1 - α_target) ≥ 1 - δ**
+
+```python
+from ssbc import ssbc_correct
+
+result = ssbc_correct(
+    alpha_target=0.10,  # Target 10% miscoverage
+    n=50,               # Calibration set size
+    delta=0.10,         # PAC parameter (90% confidence)
+    mode="beta"         # Infinite test window
+)
+
+print(f"Corrected α: {result.alpha_corrected:.4f}")
+print(f"u*: {result.u_star}")
+```
+
+### Parameters
+
+- `alpha_target`: Target miscoverage rate (e.g., 0.10 for 90% coverage)
+- `n`: Calibration set size
+- `delta`: PAC risk tolerance (probability of violating guarantee)
+- `mode`: "beta" (infinite test) or "beta-binomial" (finite test)
+
+## Module Structure
+
+The library is organized into focused modules:
+
+### Core Modules
+
+- **`ssbc.core`**: Core SSBC algorithm (`ssbc_correct`, `SSBCResult`)
+- **`ssbc.conformal`**: Mondrian conformal prediction (`mondrian_conformal_calibrate`, `split_by_class`)
+- **`ssbc.statistics`**: Statistical utilities (`clopper_pearson_intervals`, `cp_interval`)
+
+### Analysis & Visualization
+
+- **`ssbc.visualization`**: Reporting and plotting (`report_prediction_stats`, `plot_parallel_coordinates_plotly`)
+- **`ssbc.hyperparameter`**: Parameter tuning (`sweep_hyperparams_and_collect`, `sweep_and_plot_parallel_plotly`)
+
+### Testing & Simulation
+
+- **`ssbc.simulation`**: Data generators (`BinaryClassifierSimulator`)
+
+## Examples
+
+The `examples/` directory contains comprehensive demonstrations:
+
+### 1. Core SSBC Algorithm
+```bash
+python examples/ssbc_core_example.py
+```
+Demonstrates the SSBC algorithm for different calibration set sizes.
+
+### 2. Mondrian Conformal Prediction
+```bash
+python examples/mondrian_conformal_example.py
+```
+Complete workflow: simulation → calibration → reporting.
+
+### 3. Hyperparameter Sweep
+```bash
+python examples/hyperparameter_sweep_example.py
+```
+Interactive parameter tuning with parallel coordinates visualization.
+
+## Hyperparameter Tuning
+
+Sweep over α and δ values to find optimal configurations:
+
+```python
+from ssbc import sweep_and_plot_parallel_plotly
+import numpy as np
+
+# Define grid
+alpha_grid = np.arange(0.05, 0.20, 0.05)
+delta_grid = np.arange(0.05, 0.20, 0.05)
+
+# Run sweep and visualize
+df, fig = sweep_and_plot_parallel_plotly(
+    class_data=class_data,
+    alpha_0=alpha_grid, delta_0=delta_grid,
+    alpha_1=alpha_grid, delta_1=delta_grid,
+    color='err_all'  # Color by error rate
+)
+
+# Save interactive plot
+fig.write_html("sweep_results.html")
+
+# Analyze results
+print(df[['a0', 'd0', 'cov', 'sing_rate', 'err_all']].head())
+```
+
+The interactive plot allows you to:
+- Brush (select) ranges on any axis to filter configurations
+- Explore trade-offs between coverage, automation, and error rates
+- Identify Pareto-optimal hyperparameter settings
+
+## Understanding the Output
+
+### Per-Class Statistics (Conditioned on True Label)
+
+For each class, the report shows:
+- **Abstentions**: Empty prediction sets
+- **Singletons**: Confident predictions (automated decisions)
+  - Correct: True label in singleton set
+  - Incorrect: True label not in singleton set
+- **Doublets**: Both labels included (escalated to human review)
+
+### Marginal Statistics (Deployment View)
+
+Overall performance metrics ignoring true labels:
+- **Coverage**: Fraction of predictions containing the true label
+- **Singleton rate**: Fraction of confident predictions (automation level)
+- **Escalation rate**: Fraction requiring human review
+- **Error rates**: By predicted class and overall
+
+### PAC Bounds
+
+The report includes theoretical and observed singleton error rates:
+- **α'_bound**: Theoretical upper bound from PAC analysis
+- **α'_observed**: Observed error rate on calibration data
+- ✓ if observed ≤ bound (PAC guarantee satisfied)
+
+## Citation
+
+If you use SSBC in your research, please cite:
+
+```bibtex
+@software{ssbc2024,
+  author = {Zwart, Petrus H},
+  title = {SSBC: Small-Sample Beta Correction},
+  year = {2024},
+  url = {https://github.com/yourusername/ssbc}
+}
+```
+
+## Contributing
+
+Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
+
+## License
+
+MIT License - see [LICENSE](LICENSE) file for details.
+
+## Credits
+
+This package was created with [Cookiecutter](https://github.com/audreyfeldroy/cookiecutter) and the [audreyfeldroy/cookiecutter-pypackage](https://github.com/audreyfeldroy/cookiecutter-pypackage) project template.

ssbc-0.1.0.dist-info/RECORD

@@ -0,0 +1,17 @@
+ssbc/__init__.py,sha256=T5Xr2tMLW0SIsphFeAKqbjHK_cjixLA8yUgdqVK53Io,1236
+ssbc/__main__.py,sha256=Qd-f8z2Q2vpiEP2x6PBFsJrpACWDVxFKQk820MhFmHo,59
+ssbc/cli.py,sha256=i5PLSsXS3glAFH1yVdd0YJAQVDNLd8z3jPJiPo5IW8k,436
+ssbc/conformal.py,sha256=4Xq6OXJ3_vm8A177_goIYPic-Bb-qzAWLnSRJhTZMik,12340
+ssbc/core.py,sha256=2qDiMgBloevB4v62Cl_CKuGhwlYF_bUd2nI_V4dSPPo,6857
+ssbc/hyperparameter.py,sha256=e61j6koQ_l8sFvv5XRYa0x3RRPbhGIV3FxnXn1N3v28,9096
+ssbc/simulation.py,sha256=V0nOUoxdDr5tPTCn1A5PRPJpwINqSi8kqwwk9msUwIo,5158
+ssbc/ssbc.py,sha256=h0hwdogXGFqerm-5ZPeT-irPn91pCcQRjiHThXsRzEk,19
+ssbc/statistics.py,sha256=UzQe6kPBIVHsGe-NMt4nECdqqIpZRnsls1Bnl2GNBgI,4824
+ssbc/utils.py,sha256=1RxiNQM7rpegUEPuFvOlbSGesR4gnWpXr82bZQCgELM,77
+ssbc/visualization.py,sha256=aAjnDKYMPMFEyD965f_8DMccTrhWeEvtFvKuqy1Cflk,19647
+ssbc-0.1.0.dist-info/licenses/LICENSE,sha256=YOKwrV5OLHoJ_e8T4lkylNpjDdEEe4vvMG0tmUxYxco,1072
+ssbc-0.1.0.dist-info/METADATA,sha256=jFZIeDJz1p6BWVvF0985urA-2g0_jnD-ETi3z5BdWwI,8357
+ssbc-0.1.0.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
+ssbc-0.1.0.dist-info/entry_points.txt,sha256=hgp8rkP_J-wInF8y42DBMGmqJ1bDGMgHQGq12Y3tMs4,38
+ssbc-0.1.0.dist-info/top_level.txt,sha256=jkM9L2hWrag3UXW32pWlFmuB_L-G1el4oUbnMTJdDv0,5
+ssbc-0.1.0.dist-info/RECORD,,

ssbc-0.1.0.dist-info/WHEEL

@@ -0,0 +1,5 @@
+Wheel-Version: 1.0
+Generator: setuptools (80.9.0)
+Root-Is-Purelib: true
+Tag: py3-none-any
+

ssbc-0.1.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+ssbc = ssbc.cli:app

ssbc-0.1.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025, Petrus H Zwart
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ssbc-0.1.0.dist-info/top_level.txt

@@ -0,0 +1 @@
+ssbc