private-lab-bench 0.1.0__tar.gz

private_lab_bench-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Dipesh Tharu Mahato
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

private_lab_bench-0.1.0/PKG-INFO

@@ -0,0 +1,268 @@
+Metadata-Version: 2.4
+Name: private-lab-bench
+Version: 0.1.0
+Summary: Local-first evaluation for scientific machine learning on private data.
+Author: Dipesh Tharu Mahato
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/dipeshbabu/private-lab-bench
+Project-URL: Repository, https://github.com/dipeshbabu/private-lab-bench
+Project-URL: Issues, https://github.com/dipeshbabu/private-lab-bench/issues
+Project-URL: Changelog, https://github.com/dipeshbabu/private-lab-bench/blob/main/CHANGELOG.md
+Project-URL: Documentation, https://github.com/dipeshbabu/private-lab-bench/tree/main/docs
+Keywords: scientific machine learning,evaluation,private data,reproducibility,privacy auditing,benchmarking
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.21
+Requires-Dist: pandas>=1.3
+Requires-Dist: scikit-learn>=1.0
+Requires-Dist: PyYAML>=6.0
+Provides-Extra: proteingym
+Requires-Dist: biopython<2,>=1.81; extra == "proteingym"
+Provides-Extra: matbench
+Requires-Dist: matminer<0.11,>=0.9.3; extra == "matbench"
+Provides-Extra: signing
+Requires-Dist: cryptography>=41.0; extra == "signing"
+Provides-Extra: rdkit
+Requires-Dist: rdkit>=2023.9.1; extra == "rdkit"
+Provides-Extra: tdc
+Requires-Dist: PyTDC==0.4.1; extra == "tdc"
+Requires-Dist: numpy<2,>=1.26.4; extra == "tdc"
+Requires-Dist: pandas<3,>=2.1.4; extra == "tdc"
+Requires-Dist: huggingface-hub<0.26,>=0.20.3; extra == "tdc"
+Requires-Dist: setuptools<81,>=77; extra == "tdc"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: jsonschema>=4.0; extra == "dev"
+Requires-Dist: cryptography>=41.0; extra == "dev"
+Provides-Extra: docs
+Requires-Dist: mkdocs<2,>=1.6; extra == "docs"
+Requires-Dist: mkdocs-material<10,>=9.7; extra == "docs"
+Dynamic: license-file
+
+<h1 align="center">PrivateLabBench</h1>
+
+<p align="center">
+  Evaluate scientific models on private data, from your terminal.
+</p>
+
+<p align="center">
+  <a href="https://github.com/dipeshbabu/private-lab-bench/actions/workflows/ci.yml"><img alt="CI" src="https://img.shields.io/github/actions/workflow/status/dipeshbabu/private-lab-bench/ci.yml?branch=main&label=build&style=flat-square"></a>
+  <a href="https://pypi.org/project/private-lab-bench/"><img alt="PyPI" src="https://img.shields.io/pypi/v/private-lab-bench?style=flat-square"></a>
+  <a href="https://github.com/dipeshbabu/private-lab-bench/blob/main/pyproject.toml"><img alt="Python 3.10, 3.11, and 3.12" src="https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue?style=flat-square"></a>
+  <a href="LICENSE"><img alt="MIT license" src="https://img.shields.io/badge/license-MIT-green?style=flat-square"></a>
+</p>
+
+<p align="center">
+  <a href="docs/quickstart.md">Quickstart</a> ·
+  <a href="docs/prediction_tables.md">Prediction tables</a> ·
+  <a href="docs/evaluation_receipts.md">Receipts</a> ·
+  <a href="CONTRIBUTING.md">Contributing</a>
+</p>
+
+---
+
+PrivateLabBench is a local evaluation framework for scientific machine learning. Bring a prediction table, measure performance across datasets and subgroups, estimate uncertainty, and compare models on matched samples. Keep the underlying data in your environment and exchange aggregate evaluation receipts with collaborators.
+
+Evaluation runs locally without an account, hosted service, or data upload.
+
+## Installation
+
+Use **Python 3.10 or later**. Create an environment for **v0.1.0**:
+
+```sh
+python -m venv .venv
+```
+
+Activate the environment and install:
+
+```sh
+# macOS / Linux
+source .venv/bin/activate
+
+# Windows PowerShell
+.venv\Scripts\Activate.ps1
+
+python -m pip install private-lab-bench==0.1.0
+plb --help
+```
+
+The `privatelabbench` command is also available. Optional molecular featurization uses `python -m pip install 'private-lab-bench[rdkit]==0.1.0'`. See the [installation guide](docs/install.md) for source/development installs, or run the [quickstart notebook in Colab](https://colab.research.google.com/github/dipeshbabu/private-lab-bench/blob/v0.1.0/examples/notebooks/quickstart.ipynb).
+
+## Run Your First Evaluation
+
+Start with a bundled synthetic benchmark pack:
+
+```sh
+plb list-packs
+plb run-pack community-tabular-regression@1.0.0
+plb verify reports/benchmark_packs/tabular_regression_receipt.shareable.json
+```
+
+Packs include example predictions and versioned evaluation settings. They work from installed wheels and require no model downloads or training. Other packs cover molecular regression and protein binary classification.
+
+## Evaluate Your Predictions
+
+Use `prediction-table/v1` to evaluate outputs from your own model:
+
+```csv
+sample_id,target,prediction,site,batch
+s01,0.12,0.14,lab-a,b1
+s02,0.20,0.23,lab-a,b1
+s03,0.33,0.31,lab-b,b2
+s04,0.41,0.45,lab-b,b2
+```
+
+Save the table as `predictions.csv`, then run:
+
+```sh
+plb evaluate predictions.csv --slice site --slice batch
+plb verify reports/evaluation/evaluation_receipt.json
+```
+
+Regression, binary classification, and multiclass probability tables are supported. Ambiguous columns or task types require explicit flags; binary probabilities with named labels require ordered negative/positive classes. Results stay local until a reviewed release policy permits export. This four-row example is too small for the default bootstrap interval or slice minimum. See the [5-minute quickstart](docs/quickstart.md) for defaults and the [prediction-table reference](docs/prediction_tables.md) for classification schemas.
+
+## Make Runs Reproducible
+
+Store task, input, slice, uncertainty, and reporting settings in YAML:
+
+```sh
+plb init predictions.csv --slice site --out evaluation.yaml
+plb run evaluation.yaml
+plb validate-config configs/tabular_uncertainty_eval.yaml
+plb run configs/tabular_uncertainty_eval.yaml
+```
+
+Config runs write local JSON and Markdown reports, an artifact manifest, and evaluation receipts. Review the uncertainty settings and choose a sampling method appropriate to your data. See [statistical uncertainty](docs/uncertainty.md) for assumptions and limits.
+
+Use `plb list-protocols` and `plb run local.yaml --protocol plb://generic/regression@1.0.0` to apply fixed, versioned evaluation rules across private datasets. Receipts record the exact protocol digest. See [evaluation protocols](docs/protocols.md) for local definitions, package providers, and release controls.
+
+## Compare Models
+
+Compare two prediction tables containing the same sample IDs and targets:
+
+```sh
+plb compare model-a.csv model-b.csv --target target --task-type regression --metric rmse --resamples 1000 --practical-threshold 0.01
+```
+
+Paired comparisons report metric differences, bootstrap intervals, and a decision relative to your practical threshold. Optional slice comparisons help locate differences across groups. See the [paired-comparison guide](docs/paired_comparison.md).
+
+Combine compatible shared receipts with `plb aggregate lab-a.shareable.json lab-b.shareable.json --model model@revision`. The [aggregation guide](docs/aggregation.md) covers site identities, signer verification, and the metrics that support pooled point estimates.
+
+Inspect cohort changes with `plb evaluate current.csv --reference reference.csv --slice site`. [Reference/current diagnostics](docs/degradation.md) report performance degradation, slice prevalence, and selected distribution changes, with explicit uncertainty and release controls.
+
+Track the same model over repeated evaluations with [`plb trend`](docs/trends.md). It compares verified shareable receipts at declared collection times, retains uncertainty and quality decisions, and reports changing slices without needing prior raw datasets.
+
+For repeated or nested observations, declare a [sampling design](docs/sampling_design.md) to resample independent clusters and keep paired models aligned. Optional slice-test families record dependence assumptions and multiplicity adjustments.
+
+Check declared split integrity before scoring with `plb audit-design` or `plb evaluate --evaluation-design design.yaml`. [Evaluation-design audits](docs/evaluation_design.md) detect supported overlap/group/time violations and keep unavailable training evidence explicit.
+
+Evaluate real Caco2 and PGP benchmark predictions with the optional [TDC ADMET integration](docs/tdc_admet.md). It preserves the pinned official test sets and metrics, checks upstream score agreement, and supports the same evaluation protocols on private ADMET tables.
+
+The [Matbench integration](docs/matbench.md) supports composition and structure tasks with pinned official folds, units, corrected probability ROC-AUC, and verified upstream fold aggregation.
+
+The [ProteinGym integration](docs/proteingym.md) evaluates DMS substitution scores within each assay, with tied-rank Spearman and explicit protein/function aggregation.
+
+Describe evaluations of released models with the [external-validation profile](docs/external_validation.md): immutable model identity, declared evaluator relationship and dataset provenance, plus explicitly approved aggregate receipts that can be compared across labs.
+
+When predictions need to be generated inside the lab, use the [trusted local scorer](docs/trusted_scoring.md) with a reviewed numeric model, pinned digest, and explicit feature units. Its optional evaluation pipeline reuses the normal protocol and release controls.
+
+Operate a [hidden benchmark](docs/hidden_benchmarks.md) with owner-held labels, strict prediction submissions, approved result receipts, and a verified results table. Ranking requires an explicit rule in the pinned scientific protocol.
+
+## Inspect Evaluation Receipts
+
+Follow the [Caco2 model-selection case](docs/case_study.md) from a fresh environment to a paired decision, critical-group diagnosis, release check, and signed receipt. It distinguishes real public benchmark evidence from its synthetic measurement-shift proxy; independent researcher trials remain pending.
+
+`evaluation-receipt/v1` separates local context from aggregate information intended for exchange:
+
+| Artifact | Contents |
+| --- | --- |
+| `<run>_receipt.json` | Local receipt, including paths, configuration, and exact results |
+| `<run>_receipt.shareable.json` | Approved aggregate receipt, written only when the export policy allows it |
+| `<run>_receipt.md` | Human-readable receipt summary |
+
+```sh
+plb verify reports/tabular_uncertainty_demo_receipt.shareable.json
+```
+
+Shareable receipts require an explicit `release-policy/v1` policy. A blocked export leaves local results and a release audit, and removes any previous shareable file. The bundled synthetic packs include reviewed example policies; private-data runs need their own review. Use `plb release-check <local-receipt.json> --policy release.yaml` for a dry run.
+
+Verification checks artifact integrity and optional signatures. A matching self-contained hash alone does not establish authorship, correct execution, or scientific validity. See [evaluation receipts](docs/evaluation_receipts.md) and [report integrity](docs/report_integrity.md).
+
+For offline signer authentication, use optional Ed25519 signatures and pinned public keys:
+
+```sh
+plb verify receipt.shareable.json --trust-store trusted-labs.yaml --require-trusted-signer --json
+```
+
+See the [receipt trust guide](docs/receipt_trust.md) for signing, key rotation, protocol binding, and the limits of these checks.
+
+## Privacy and Sharing
+
+Keeping data local limits data movement; aggregate results can still disclose sensitive information. PrivateLabBench provides separate tools for exact reporting, empirical privacy audits, release-policy checks, and heuristic `metric_perturbation`.
+
+Metric perturbation, including the deprecated `dp` alias, **does not provide a formal differential privacy guarantee**. Bounded-query DP reference primitives have their own sensitivity and accounting requirements. Read the [privacy guide](docs/privacy.md) before selecting a release method.
+
+## Reference
+
+Enforce explicit model thresholds in CI with `plb gate <receipt> --policy quality.yaml`.
+See [quality gates](docs/quality_gates.md) for confidence, baseline, slice, and privacy requirements.
+
+Use `--help` to explore commands and options:
+
+```sh
+plb eval-predictions --help
+plb compare --help
+plb list-tasks
+plb list-privacy-attacks
+```
+
+- [Five-minute quickstart](docs/quickstart.md)
+- [Core concepts](docs/concepts.md)
+- [Tasks and plugins](docs/tasks.md)
+- [Community benchmark packs](docs/benchmark_packs.md)
+- [Troubleshooting](docs/troubleshooting.md)
+- [Project scope](docs/project_scope.md) and [roadmap](docs/roadmap.md)
+
+## Developing
+
+Install the development and documentation tools:
+
+```sh
+python -m pip install -e '.[dev,docs]' build twine
+python -m pytest -q
+python -m build
+python -m twine check dist/*
+mkdocs build
+```
+
+CI tests Python 3.10, 3.11, and 3.12, validates packages and documentation, and exercises the Docker runner. It also installs the wheel in a clean environment and runs a bundled pack outside the checkout. Release publishing uses the [tag-based release workflow](docs/releases.md).
+
+Useful source entry points:
+
+| Path | Purpose |
+| --- | --- |
+| `privatelabbench/cli.py` | Command-line interface |
+| `privatelabbench/core/` | Task registry and extension interfaces |
+| `privatelabbench/eval/` | Metrics, slices, uncertainty, and paired comparisons |
+| `privatelabbench/privacy/` | Privacy audits and release policies |
+| `privatelabbench/reports/` | Reports, receipts, and artifact integrity |
+| `privatelabbench/data/` | Bundled benchmark packs and schemas |
+
+## Contributing
+
+Focused fixes, scientific-domain plugins, benchmark packs, and documentation improvements are welcome. Read [CONTRIBUTING.md](CONTRIBUTING.md) for setup, extension boundaries, and validation requirements. Discuss new tasks, privacy methods, or schema changes in an issue before implementation.
+
+The project follows its [Code of Conduct](CODE_OF_CONDUCT.md) and [governance guide](GOVERNANCE.md). Report sensitive vulnerabilities through the process in [SECURITY.md](SECURITY.md).
+
+## License
+
+Released under the [MIT license](LICENSE). Use [CITATION.cff](CITATION.cff) to cite PrivateLabBench in research.

private_lab_bench-0.1.0/README.md

@@ -0,0 +1,218 @@
+<h1 align="center">PrivateLabBench</h1>
+
+<p align="center">
+  Evaluate scientific models on private data, from your terminal.
+</p>
+
+<p align="center">
+  <a href="https://github.com/dipeshbabu/private-lab-bench/actions/workflows/ci.yml"><img alt="CI" src="https://img.shields.io/github/actions/workflow/status/dipeshbabu/private-lab-bench/ci.yml?branch=main&label=build&style=flat-square"></a>
+  <a href="https://pypi.org/project/private-lab-bench/"><img alt="PyPI" src="https://img.shields.io/pypi/v/private-lab-bench?style=flat-square"></a>
+  <a href="https://github.com/dipeshbabu/private-lab-bench/blob/main/pyproject.toml"><img alt="Python 3.10, 3.11, and 3.12" src="https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue?style=flat-square"></a>
+  <a href="LICENSE"><img alt="MIT license" src="https://img.shields.io/badge/license-MIT-green?style=flat-square"></a>
+</p>
+
+<p align="center">
+  <a href="docs/quickstart.md">Quickstart</a> ·
+  <a href="docs/prediction_tables.md">Prediction tables</a> ·
+  <a href="docs/evaluation_receipts.md">Receipts</a> ·
+  <a href="CONTRIBUTING.md">Contributing</a>
+</p>
+
+---
+
+PrivateLabBench is a local evaluation framework for scientific machine learning. Bring a prediction table, measure performance across datasets and subgroups, estimate uncertainty, and compare models on matched samples. Keep the underlying data in your environment and exchange aggregate evaluation receipts with collaborators.
+
+Evaluation runs locally without an account, hosted service, or data upload.
+
+## Installation
+
+Use **Python 3.10 or later**. Create an environment for **v0.1.0**:
+
+```sh
+python -m venv .venv
+```
+
+Activate the environment and install:
+
+```sh
+# macOS / Linux
+source .venv/bin/activate
+
+# Windows PowerShell
+.venv\Scripts\Activate.ps1
+
+python -m pip install private-lab-bench==0.1.0
+plb --help
+```
+
+The `privatelabbench` command is also available. Optional molecular featurization uses `python -m pip install 'private-lab-bench[rdkit]==0.1.0'`. See the [installation guide](docs/install.md) for source/development installs, or run the [quickstart notebook in Colab](https://colab.research.google.com/github/dipeshbabu/private-lab-bench/blob/v0.1.0/examples/notebooks/quickstart.ipynb).
+
+## Run Your First Evaluation
+
+Start with a bundled synthetic benchmark pack:
+
+```sh
+plb list-packs
+plb run-pack community-tabular-regression@1.0.0
+plb verify reports/benchmark_packs/tabular_regression_receipt.shareable.json
+```
+
+Packs include example predictions and versioned evaluation settings. They work from installed wheels and require no model downloads or training. Other packs cover molecular regression and protein binary classification.
+
+## Evaluate Your Predictions
+
+Use `prediction-table/v1` to evaluate outputs from your own model:
+
+```csv
+sample_id,target,prediction,site,batch
+s01,0.12,0.14,lab-a,b1
+s02,0.20,0.23,lab-a,b1
+s03,0.33,0.31,lab-b,b2
+s04,0.41,0.45,lab-b,b2
+```
+
+Save the table as `predictions.csv`, then run:
+
+```sh
+plb evaluate predictions.csv --slice site --slice batch
+plb verify reports/evaluation/evaluation_receipt.json
+```
+
+Regression, binary classification, and multiclass probability tables are supported. Ambiguous columns or task types require explicit flags; binary probabilities with named labels require ordered negative/positive classes. Results stay local until a reviewed release policy permits export. This four-row example is too small for the default bootstrap interval or slice minimum. See the [5-minute quickstart](docs/quickstart.md) for defaults and the [prediction-table reference](docs/prediction_tables.md) for classification schemas.
+
+## Make Runs Reproducible
+
+Store task, input, slice, uncertainty, and reporting settings in YAML:
+
+```sh
+plb init predictions.csv --slice site --out evaluation.yaml
+plb run evaluation.yaml
+plb validate-config configs/tabular_uncertainty_eval.yaml
+plb run configs/tabular_uncertainty_eval.yaml
+```
+
+Config runs write local JSON and Markdown reports, an artifact manifest, and evaluation receipts. Review the uncertainty settings and choose a sampling method appropriate to your data. See [statistical uncertainty](docs/uncertainty.md) for assumptions and limits.
+
+Use `plb list-protocols` and `plb run local.yaml --protocol plb://generic/regression@1.0.0` to apply fixed, versioned evaluation rules across private datasets. Receipts record the exact protocol digest. See [evaluation protocols](docs/protocols.md) for local definitions, package providers, and release controls.
+
+## Compare Models
+
+Compare two prediction tables containing the same sample IDs and targets:
+
+```sh
+plb compare model-a.csv model-b.csv --target target --task-type regression --metric rmse --resamples 1000 --practical-threshold 0.01
+```
+
+Paired comparisons report metric differences, bootstrap intervals, and a decision relative to your practical threshold. Optional slice comparisons help locate differences across groups. See the [paired-comparison guide](docs/paired_comparison.md).
+
+Combine compatible shared receipts with `plb aggregate lab-a.shareable.json lab-b.shareable.json --model model@revision`. The [aggregation guide](docs/aggregation.md) covers site identities, signer verification, and the metrics that support pooled point estimates.
+
+Inspect cohort changes with `plb evaluate current.csv --reference reference.csv --slice site`. [Reference/current diagnostics](docs/degradation.md) report performance degradation, slice prevalence, and selected distribution changes, with explicit uncertainty and release controls.
+
+Track the same model over repeated evaluations with [`plb trend`](docs/trends.md). It compares verified shareable receipts at declared collection times, retains uncertainty and quality decisions, and reports changing slices without needing prior raw datasets.
+
+For repeated or nested observations, declare a [sampling design](docs/sampling_design.md) to resample independent clusters and keep paired models aligned. Optional slice-test families record dependence assumptions and multiplicity adjustments.
+
+Check declared split integrity before scoring with `plb audit-design` or `plb evaluate --evaluation-design design.yaml`. [Evaluation-design audits](docs/evaluation_design.md) detect supported overlap/group/time violations and keep unavailable training evidence explicit.
+
+Evaluate real Caco2 and PGP benchmark predictions with the optional [TDC ADMET integration](docs/tdc_admet.md). It preserves the pinned official test sets and metrics, checks upstream score agreement, and supports the same evaluation protocols on private ADMET tables.
+
+The [Matbench integration](docs/matbench.md) supports composition and structure tasks with pinned official folds, units, corrected probability ROC-AUC, and verified upstream fold aggregation.
+
+The [ProteinGym integration](docs/proteingym.md) evaluates DMS substitution scores within each assay, with tied-rank Spearman and explicit protein/function aggregation.
+
+Describe evaluations of released models with the [external-validation profile](docs/external_validation.md): immutable model identity, declared evaluator relationship and dataset provenance, plus explicitly approved aggregate receipts that can be compared across labs.
+
+When predictions need to be generated inside the lab, use the [trusted local scorer](docs/trusted_scoring.md) with a reviewed numeric model, pinned digest, and explicit feature units. Its optional evaluation pipeline reuses the normal protocol and release controls.
+
+Operate a [hidden benchmark](docs/hidden_benchmarks.md) with owner-held labels, strict prediction submissions, approved result receipts, and a verified results table. Ranking requires an explicit rule in the pinned scientific protocol.
+
+## Inspect Evaluation Receipts
+
+Follow the [Caco2 model-selection case](docs/case_study.md) from a fresh environment to a paired decision, critical-group diagnosis, release check, and signed receipt. It distinguishes real public benchmark evidence from its synthetic measurement-shift proxy; independent researcher trials remain pending.
+
+`evaluation-receipt/v1` separates local context from aggregate information intended for exchange:
+
+| Artifact | Contents |
+| --- | --- |
+| `<run>_receipt.json` | Local receipt, including paths, configuration, and exact results |
+| `<run>_receipt.shareable.json` | Approved aggregate receipt, written only when the export policy allows it |
+| `<run>_receipt.md` | Human-readable receipt summary |
+
+```sh
+plb verify reports/tabular_uncertainty_demo_receipt.shareable.json
+```
+
+Shareable receipts require an explicit `release-policy/v1` policy. A blocked export leaves local results and a release audit, and removes any previous shareable file. The bundled synthetic packs include reviewed example policies; private-data runs need their own review. Use `plb release-check <local-receipt.json> --policy release.yaml` for a dry run.
+
+Verification checks artifact integrity and optional signatures. A matching self-contained hash alone does not establish authorship, correct execution, or scientific validity. See [evaluation receipts](docs/evaluation_receipts.md) and [report integrity](docs/report_integrity.md).
+
+For offline signer authentication, use optional Ed25519 signatures and pinned public keys:
+
+```sh
+plb verify receipt.shareable.json --trust-store trusted-labs.yaml --require-trusted-signer --json
+```
+
+See the [receipt trust guide](docs/receipt_trust.md) for signing, key rotation, protocol binding, and the limits of these checks.
+
+## Privacy and Sharing
+
+Keeping data local limits data movement; aggregate results can still disclose sensitive information. PrivateLabBench provides separate tools for exact reporting, empirical privacy audits, release-policy checks, and heuristic `metric_perturbation`.
+
+Metric perturbation, including the deprecated `dp` alias, **does not provide a formal differential privacy guarantee**. Bounded-query DP reference primitives have their own sensitivity and accounting requirements. Read the [privacy guide](docs/privacy.md) before selecting a release method.
+
+## Reference
+
+Enforce explicit model thresholds in CI with `plb gate <receipt> --policy quality.yaml`.
+See [quality gates](docs/quality_gates.md) for confidence, baseline, slice, and privacy requirements.
+
+Use `--help` to explore commands and options:
+
+```sh
+plb eval-predictions --help
+plb compare --help
+plb list-tasks
+plb list-privacy-attacks
+```
+
+- [Five-minute quickstart](docs/quickstart.md)
+- [Core concepts](docs/concepts.md)
+- [Tasks and plugins](docs/tasks.md)
+- [Community benchmark packs](docs/benchmark_packs.md)
+- [Troubleshooting](docs/troubleshooting.md)
+- [Project scope](docs/project_scope.md) and [roadmap](docs/roadmap.md)
+
+## Developing
+
+Install the development and documentation tools:
+
+```sh
+python -m pip install -e '.[dev,docs]' build twine
+python -m pytest -q
+python -m build
+python -m twine check dist/*
+mkdocs build
+```
+
+CI tests Python 3.10, 3.11, and 3.12, validates packages and documentation, and exercises the Docker runner. It also installs the wheel in a clean environment and runs a bundled pack outside the checkout. Release publishing uses the [tag-based release workflow](docs/releases.md).
+
+Useful source entry points:
+
+| Path | Purpose |
+| --- | --- |
+| `privatelabbench/cli.py` | Command-line interface |
+| `privatelabbench/core/` | Task registry and extension interfaces |
+| `privatelabbench/eval/` | Metrics, slices, uncertainty, and paired comparisons |
+| `privatelabbench/privacy/` | Privacy audits and release policies |
+| `privatelabbench/reports/` | Reports, receipts, and artifact integrity |
+| `privatelabbench/data/` | Bundled benchmark packs and schemas |
+
+## Contributing
+
+Focused fixes, scientific-domain plugins, benchmark packs, and documentation improvements are welcome. Read [CONTRIBUTING.md](CONTRIBUTING.md) for setup, extension boundaries, and validation requirements. Discuss new tasks, privacy methods, or schema changes in an issue before implementation.
+
+The project follows its [Code of Conduct](CODE_OF_CONDUCT.md) and [governance guide](GOVERNANCE.md). Report sensitive vulnerabilities through the process in [SECURITY.md](SECURITY.md).
+
+## License
+
+Released under the [MIT license](LICENSE). Use [CITATION.cff](CITATION.cff) to cite PrivateLabBench in research.