abkit 0.1.0__tar.gz → 0.1.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (31) hide show
  1. {abkit-0.1.0/src/abkit.egg-info → abkit-0.1.1}/PKG-INFO +17 -8
  2. {abkit-0.1.0 → abkit-0.1.1}/README.md +15 -6
  3. {abkit-0.1.0 → abkit-0.1.1}/pyproject.toml +2 -2
  4. {abkit-0.1.0 → abkit-0.1.1}/src/abkit/__init__.py +2 -2
  5. {abkit-0.1.0 → abkit-0.1.1}/src/abkit/cli.py +3 -1
  6. {abkit-0.1.0 → abkit-0.1.1/src/abkit.egg-info}/PKG-INFO +17 -8
  7. {abkit-0.1.0 → abkit-0.1.1}/LICENSE +0 -0
  8. {abkit-0.1.0 → abkit-0.1.1}/setup.cfg +0 -0
  9. {abkit-0.1.0 → abkit-0.1.1}/src/abkit/audit.py +0 -0
  10. {abkit-0.1.0 → abkit-0.1.1}/src/abkit/bootstrap.py +0 -0
  11. {abkit-0.1.0 → abkit-0.1.1}/src/abkit/demo.py +0 -0
  12. {abkit-0.1.0 → abkit-0.1.1}/src/abkit/io.py +0 -0
  13. {abkit-0.1.0 → abkit-0.1.1}/src/abkit/probes.py +0 -0
  14. {abkit-0.1.0 → abkit-0.1.1}/src/abkit/py.typed +0 -0
  15. {abkit-0.1.0 → abkit-0.1.1}/src/abkit/report.py +0 -0
  16. {abkit-0.1.0 → abkit-0.1.1}/src/abkit/result.py +0 -0
  17. {abkit-0.1.0 → abkit-0.1.1}/src/abkit/schema.py +0 -0
  18. {abkit-0.1.0 → abkit-0.1.1}/src/abkit/stats.py +0 -0
  19. {abkit-0.1.0 → abkit-0.1.1}/src/abkit/synthetic.py +0 -0
  20. {abkit-0.1.0 → abkit-0.1.1}/src/abkit.egg-info/SOURCES.txt +0 -0
  21. {abkit-0.1.0 → abkit-0.1.1}/src/abkit.egg-info/dependency_links.txt +0 -0
  22. {abkit-0.1.0 → abkit-0.1.1}/src/abkit.egg-info/entry_points.txt +0 -0
  23. {abkit-0.1.0 → abkit-0.1.1}/src/abkit.egg-info/requires.txt +0 -0
  24. {abkit-0.1.0 → abkit-0.1.1}/src/abkit.egg-info/top_level.txt +0 -0
  25. {abkit-0.1.0 → abkit-0.1.1}/tests/test_bootstrap.py +0 -0
  26. {abkit-0.1.0 → abkit-0.1.1}/tests/test_cli.py +0 -0
  27. {abkit-0.1.0 → abkit-0.1.1}/tests/test_probes.py +0 -0
  28. {abkit-0.1.0 → abkit-0.1.1}/tests/test_result.py +0 -0
  29. {abkit-0.1.0 → abkit-0.1.1}/tests/test_schema.py +0 -0
  30. {abkit-0.1.0 → abkit-0.1.1}/tests/test_stats.py +0 -0
  31. {abkit-0.1.0 → abkit-0.1.1}/tests/test_synthetic.py +0 -0
@@ -1,7 +1,7 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: abkit
3
- Version: 0.1.0
4
- Summary: Audit an A/B-test readout before you ship the decision
3
+ Version: 0.1.1
4
+ Summary: Sample-ratio, peeking, multiple-testing and winner's-curse checks for experiment readouts
5
5
  Author: Hadi Mohammadi
6
6
  License: MIT
7
7
  Project-URL: Homepage, https://github.com/mohammadi-hadi/abkit
@@ -30,9 +30,12 @@ Dynamic: license-file
30
30
 
31
31
  # abkit
32
32
 
33
- Audit an A/B-test readout before you ship the decision.
33
+ Sample-ratio, peeking, multiple-testing and winner's-curse checks for experiment readouts.
34
34
 
35
35
  [![ci](https://github.com/mohammadi-hadi/abkit/actions/workflows/ci.yml/badge.svg)](https://github.com/mohammadi-hadi/abkit/actions/workflows/ci.yml)
36
+ [![PyPI](https://img.shields.io/pypi/v/abkit)](https://pypi.org/project/abkit/)
37
+ [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.21809478.svg)](https://doi.org/10.5281/zenodo.21809478)
38
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
36
39
 
37
40
  An experiment readout can say "B wins, p = 0.03, ship it" while the traffic
38
41
  split was broken, three whale customers carried the whole effect, the analyst
@@ -96,7 +99,7 @@ the exploratory split of the [Upworthy Research Archive](https://doi.org/10.1038
96
99
  ## Install
97
100
 
98
101
  ```
99
- pip install git+https://github.com/mohammadi-hadi/abkit
102
+ pip install abkit
100
103
  ```
101
104
 
102
105
  Python 3.11+. Runtime dependencies: numpy, pydantic, matplotlib.
@@ -239,19 +242,25 @@ uncertainty on every number and validation by implantation.
239
242
 
240
243
  Companion projects: [judgekit](https://github.com/mohammadi-hadi/judgekit)
241
244
  applies the same audit-before-you-trust pattern to LLM judges;
242
- [trajectory-judge](https://github.com/mohammadi-hadi/trajectory-judge)
243
- measures what outcome-only judges miss on agent trajectories.
245
+ [raterkit](https://github.com/mohammadi-hadi/raterkit) applies it to the human
246
+ labels underneath both; [trajectory-judge](https://github.com/mohammadi-hadi/trajectory-judge)
247
+ measures what outcome-only judges miss on agent trajectories;
248
+ [arenakit](https://github.com/mohammadi-hadi/arenakit) applies it to pairwise
249
+ model leaderboards.
244
250
 
245
251
  ## Citation
246
252
 
247
- If abkit is useful in your work, please cite it (see
248
- [CITATION.cff](CITATION.cff)):
253
+ Releases are archived on Zenodo. Cite the concept DOI
254
+ [10.5281/zenodo.21809478](https://doi.org/10.5281/zenodo.21809478), which always
255
+ resolves to the latest version; structured metadata is in
256
+ [CITATION.cff](CITATION.cff).
249
257
 
250
258
  ```bibtex
251
259
  @software{mohammadi_abkit,
252
260
  author = {Mohammadi, Hadi},
253
261
  title = {abkit: audit an A/B-test readout before you ship the decision},
254
262
  url = {https://github.com/mohammadi-hadi/abkit},
263
+ doi = {10.5281/zenodo.21809478},
255
264
  version = {0.1.0},
256
265
  year = {2026}
257
266
  }
@@ -1,8 +1,11 @@
1
1
  # abkit
2
2
 
3
- Audit an A/B-test readout before you ship the decision.
3
+ Sample-ratio, peeking, multiple-testing and winner's-curse checks for experiment readouts.
4
4
 
5
5
  [![ci](https://github.com/mohammadi-hadi/abkit/actions/workflows/ci.yml/badge.svg)](https://github.com/mohammadi-hadi/abkit/actions/workflows/ci.yml)
6
+ [![PyPI](https://img.shields.io/pypi/v/abkit)](https://pypi.org/project/abkit/)
7
+ [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.21809478.svg)](https://doi.org/10.5281/zenodo.21809478)
8
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
6
9
 
7
10
  An experiment readout can say "B wins, p = 0.03, ship it" while the traffic
8
11
  split was broken, three whale customers carried the whole effect, the analyst
@@ -66,7 +69,7 @@ the exploratory split of the [Upworthy Research Archive](https://doi.org/10.1038
66
69
  ## Install
67
70
 
68
71
  ```
69
- pip install git+https://github.com/mohammadi-hadi/abkit
72
+ pip install abkit
70
73
  ```
71
74
 
72
75
  Python 3.11+. Runtime dependencies: numpy, pydantic, matplotlib.
@@ -209,19 +212,25 @@ uncertainty on every number and validation by implantation.
209
212
 
210
213
  Companion projects: [judgekit](https://github.com/mohammadi-hadi/judgekit)
211
214
  applies the same audit-before-you-trust pattern to LLM judges;
212
- [trajectory-judge](https://github.com/mohammadi-hadi/trajectory-judge)
213
- measures what outcome-only judges miss on agent trajectories.
215
+ [raterkit](https://github.com/mohammadi-hadi/raterkit) applies it to the human
216
+ labels underneath both; [trajectory-judge](https://github.com/mohammadi-hadi/trajectory-judge)
217
+ measures what outcome-only judges miss on agent trajectories;
218
+ [arenakit](https://github.com/mohammadi-hadi/arenakit) applies it to pairwise
219
+ model leaderboards.
214
220
 
215
221
  ## Citation
216
222
 
217
- If abkit is useful in your work, please cite it (see
218
- [CITATION.cff](CITATION.cff)):
223
+ Releases are archived on Zenodo. Cite the concept DOI
224
+ [10.5281/zenodo.21809478](https://doi.org/10.5281/zenodo.21809478), which always
225
+ resolves to the latest version; structured metadata is in
226
+ [CITATION.cff](CITATION.cff).
219
227
 
220
228
  ```bibtex
221
229
  @software{mohammadi_abkit,
222
230
  author = {Mohammadi, Hadi},
223
231
  title = {abkit: audit an A/B-test readout before you ship the decision},
224
232
  url = {https://github.com/mohammadi-hadi/abkit},
233
+ doi = {10.5281/zenodo.21809478},
225
234
  version = {0.1.0},
226
235
  year = {2026}
227
236
  }
@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "abkit"
7
- version = "0.1.0"
8
- description = "Audit an A/B-test readout before you ship the decision"
7
+ version = "0.1.1"
8
+ description = "Sample-ratio, peeking, multiple-testing and winner's-curse checks for experiment readouts"
9
9
  readme = "README.md"
10
10
  license = {text = "MIT"}
11
11
  authors = [{name = "Hadi Mohammadi"}]
@@ -1,11 +1,11 @@
1
- """abkit: audit an A/B-test readout before you ship the decision."""
1
+ """abkit: sample-ratio, peeking, multiple-testing and winner's-curse checks."""
2
2
 
3
3
  from .audit import run_audit
4
4
  from .io import dump_design, dump_units, load_design, load_units
5
5
  from .result import AuditResult, ProbeResult, SkippedProbe
6
6
  from .schema import Design, Look, Unit
7
7
 
8
- __version__ = "0.1.0"
8
+ __version__ = "0.1.1"
9
9
 
10
10
  __all__ = [
11
11
  "AuditResult",
@@ -14,7 +14,9 @@ from .report import inject_readme, write_report
14
14
 
15
15
  def main(argv: list[str] | None = None) -> int:
16
16
  parser = argparse.ArgumentParser(
17
- prog="abkit", description="Audit an A/B-test readout before you ship the decision."
17
+ prog="abkit",
18
+ description="Sample-ratio, peeking, multiple-testing and winner's-curse checks "
19
+ "for experiment readouts.",
18
20
  )
19
21
  parser.add_argument("--version", action="version", version=f"abkit {__version__}")
20
22
  sub = parser.add_subparsers(dest="command", required=True)
@@ -1,7 +1,7 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: abkit
3
- Version: 0.1.0
4
- Summary: Audit an A/B-test readout before you ship the decision
3
+ Version: 0.1.1
4
+ Summary: Sample-ratio, peeking, multiple-testing and winner's-curse checks for experiment readouts
5
5
  Author: Hadi Mohammadi
6
6
  License: MIT
7
7
  Project-URL: Homepage, https://github.com/mohammadi-hadi/abkit
@@ -30,9 +30,12 @@ Dynamic: license-file
30
30
 
31
31
  # abkit
32
32
 
33
- Audit an A/B-test readout before you ship the decision.
33
+ Sample-ratio, peeking, multiple-testing and winner's-curse checks for experiment readouts.
34
34
 
35
35
  [![ci](https://github.com/mohammadi-hadi/abkit/actions/workflows/ci.yml/badge.svg)](https://github.com/mohammadi-hadi/abkit/actions/workflows/ci.yml)
36
+ [![PyPI](https://img.shields.io/pypi/v/abkit)](https://pypi.org/project/abkit/)
37
+ [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.21809478.svg)](https://doi.org/10.5281/zenodo.21809478)
38
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
36
39
 
37
40
  An experiment readout can say "B wins, p = 0.03, ship it" while the traffic
38
41
  split was broken, three whale customers carried the whole effect, the analyst
@@ -96,7 +99,7 @@ the exploratory split of the [Upworthy Research Archive](https://doi.org/10.1038
96
99
  ## Install
97
100
 
98
101
  ```
99
- pip install git+https://github.com/mohammadi-hadi/abkit
102
+ pip install abkit
100
103
  ```
101
104
 
102
105
  Python 3.11+. Runtime dependencies: numpy, pydantic, matplotlib.
@@ -239,19 +242,25 @@ uncertainty on every number and validation by implantation.
239
242
 
240
243
  Companion projects: [judgekit](https://github.com/mohammadi-hadi/judgekit)
241
244
  applies the same audit-before-you-trust pattern to LLM judges;
242
- [trajectory-judge](https://github.com/mohammadi-hadi/trajectory-judge)
243
- measures what outcome-only judges miss on agent trajectories.
245
+ [raterkit](https://github.com/mohammadi-hadi/raterkit) applies it to the human
246
+ labels underneath both; [trajectory-judge](https://github.com/mohammadi-hadi/trajectory-judge)
247
+ measures what outcome-only judges miss on agent trajectories;
248
+ [arenakit](https://github.com/mohammadi-hadi/arenakit) applies it to pairwise
249
+ model leaderboards.
244
250
 
245
251
  ## Citation
246
252
 
247
- If abkit is useful in your work, please cite it (see
248
- [CITATION.cff](CITATION.cff)):
253
+ Releases are archived on Zenodo. Cite the concept DOI
254
+ [10.5281/zenodo.21809478](https://doi.org/10.5281/zenodo.21809478), which always
255
+ resolves to the latest version; structured metadata is in
256
+ [CITATION.cff](CITATION.cff).
249
257
 
250
258
  ```bibtex
251
259
  @software{mohammadi_abkit,
252
260
  author = {Mohammadi, Hadi},
253
261
  title = {abkit: audit an A/B-test readout before you ship the decision},
254
262
  url = {https://github.com/mohammadi-hadi/abkit},
263
+ doi = {10.5281/zenodo.21809478},
255
264
  version = {0.1.0},
256
265
  year = {2026}
257
266
  }
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes