repro-evidence-kit 0.4.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- repro_evidence_kit-0.4.1/.github/ISSUE_TEMPLATE/bug_report.md +27 -0
- repro_evidence_kit-0.4.1/.github/ISSUE_TEMPLATE/feature_request.md +22 -0
- repro_evidence_kit-0.4.1/.github/pull_request_template.md +13 -0
- repro_evidence_kit-0.4.1/.github/workflows/ci.yml +39 -0
- repro_evidence_kit-0.4.1/.github/workflows/publish.yml +46 -0
- repro_evidence_kit-0.4.1/.gitignore +12 -0
- repro_evidence_kit-0.4.1/CHANGELOG.md +49 -0
- repro_evidence_kit-0.4.1/CONTRIBUTING.md +12 -0
- repro_evidence_kit-0.4.1/LICENSE +17 -0
- repro_evidence_kit-0.4.1/PKG-INFO +153 -0
- repro_evidence_kit-0.4.1/README.md +124 -0
- repro_evidence_kit-0.4.1/ROADMAP.md +38 -0
- repro_evidence_kit-0.4.1/SECURITY.md +5 -0
- repro_evidence_kit-0.4.1/docs/cli-exit-codes.md +11 -0
- repro_evidence_kit-0.4.1/docs/cli.md +123 -0
- repro_evidence_kit-0.4.1/docs/design-principles.md +7 -0
- repro_evidence_kit-0.4.1/docs/evidence-bundle-format.md +36 -0
- repro_evidence_kit-0.4.1/docs/github-actions.md +260 -0
- repro_evidence_kit-0.4.1/docs/maintainer-workflow.md +52 -0
- repro_evidence_kit-0.4.1/docs/publishing.md +42 -0
- repro_evidence_kit-0.4.1/docs/release-checklist.md +41 -0
- repro_evidence_kit-0.4.1/docs/signed-bundles.md +102 -0
- repro_evidence_kit-0.4.1/docs/tutorial.md +83 -0
- repro_evidence_kit-0.4.1/docs/use-cases.md +19 -0
- repro_evidence_kit-0.4.1/examples/dummy-binary/notes.txt +1 -0
- repro_evidence_kit-0.4.1/examples/dummy-binary/sample.bin +0 -0
- repro_evidence_kit-0.4.1/examples/evidence-bundle.yaml +23 -0
- repro_evidence_kit-0.4.1/examples/sandbox-run/input.txt +1 -0
- repro_evidence_kit-0.4.1/examples/sandbox-run/result.txt +1 -0
- repro_evidence_kit-0.4.1/examples/signed-bundle/README.md +20 -0
- repro_evidence_kit-0.4.1/pyproject.toml +40 -0
- repro_evidence_kit-0.4.1/schemas/evidence-bundle.schema.json +38 -0
- repro_evidence_kit-0.4.1/schemas/signature-sidecar.schema.json +42 -0
- repro_evidence_kit-0.4.1/scripts/check_dist.py +36 -0
- repro_evidence_kit-0.4.1/scripts/generate_fixtures.py +38 -0
- repro_evidence_kit-0.4.1/scripts/release_install_smoke.py +57 -0
- repro_evidence_kit-0.4.1/scripts/smoke_examples.py +75 -0
- repro_evidence_kit-0.4.1/src/repro_evidence_kit/__init__.py +3 -0
- repro_evidence_kit-0.4.1/src/repro_evidence_kit/__main__.py +3 -0
- repro_evidence_kit-0.4.1/src/repro_evidence_kit/cli.py +165 -0
- repro_evidence_kit-0.4.1/src/repro_evidence_kit/evidence.py +143 -0
- repro_evidence_kit-0.4.1/src/repro_evidence_kit/manifest.py +184 -0
- repro_evidence_kit-0.4.1/src/repro_evidence_kit/schemas/__init__.py +0 -0
- repro_evidence_kit-0.4.1/src/repro_evidence_kit/schemas/evidence-bundle.schema.json +38 -0
- repro_evidence_kit-0.4.1/src/repro_evidence_kit/schemas/signature-sidecar.schema.json +42 -0
- repro_evidence_kit-0.4.1/src/repro_evidence_kit/signing.py +140 -0
- repro_evidence_kit-0.4.1/src/repro_evidence_kit/verify.py +128 -0
- repro_evidence_kit-0.4.1/tests/test_cli.py +331 -0
- repro_evidence_kit-0.4.1/tests/test_evidence.py +94 -0
- repro_evidence_kit-0.4.1/tests/test_manifest.py +85 -0
- repro_evidence_kit-0.4.1/tests/test_signing.py +95 -0
- repro_evidence_kit-0.4.1/tests/test_verify.py +68 -0
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Bug report
|
|
3
|
+
about: Report incorrect CLI, manifest, verifier, or schema behavior
|
|
4
|
+
title: "bug: "
|
|
5
|
+
labels: bug
|
|
6
|
+
assignees: ''
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## What happened?
|
|
10
|
+
|
|
11
|
+
## Expected behavior
|
|
12
|
+
|
|
13
|
+
## Reproduction steps
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
# commands here
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
## Environment
|
|
20
|
+
|
|
21
|
+
- OS:
|
|
22
|
+
- Python version:
|
|
23
|
+
- repro-evidence-kit version:
|
|
24
|
+
|
|
25
|
+
## Data policy confirmation
|
|
26
|
+
|
|
27
|
+
- [ ] This issue does not include proprietary binaries, private datasets, credentials, sensitive logs, or target-specific private artifacts.
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Feature request
|
|
3
|
+
about: Suggest a general-purpose reproducibility or evidence workflow improvement
|
|
4
|
+
title: "feature: "
|
|
5
|
+
labels: enhancement
|
|
6
|
+
assignees: ''
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Problem
|
|
10
|
+
|
|
11
|
+
## Proposed behavior
|
|
12
|
+
|
|
13
|
+
## Example workflow
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
# optional commands
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
## Scope check
|
|
20
|
+
|
|
21
|
+
- [ ] This feature is general-purpose and not tied to one private target or proprietary dataset.
|
|
22
|
+
- [ ] This feature can be tested with synthetic or redistributable fixtures.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
## Summary
|
|
2
|
+
|
|
3
|
+
## Tests
|
|
4
|
+
|
|
5
|
+
- [ ] `python -m unittest discover -s tests`
|
|
6
|
+
- [ ] `python -m repro_evidence_kit evidence validate examples/evidence-bundle.yaml`
|
|
7
|
+
|
|
8
|
+
## Data policy
|
|
9
|
+
|
|
10
|
+
- [ ] No proprietary binaries, private datasets, credentials, sensitive logs, or target-specific private artifacts are included.
|
|
11
|
+
- [ ] New examples are synthetic or clearly redistributable.
|
|
12
|
+
|
|
13
|
+
## Notes for maintainers
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
name: CI
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
pull_request:
|
|
6
|
+
|
|
7
|
+
jobs:
|
|
8
|
+
test:
|
|
9
|
+
name: Python ${{ matrix.python-version }}
|
|
10
|
+
runs-on: ubuntu-latest
|
|
11
|
+
strategy:
|
|
12
|
+
fail-fast: false
|
|
13
|
+
matrix:
|
|
14
|
+
python-version: ['3.10', '3.11', '3.12']
|
|
15
|
+
steps:
|
|
16
|
+
- uses: actions/checkout@v4
|
|
17
|
+
- uses: actions/setup-python@v5
|
|
18
|
+
with:
|
|
19
|
+
python-version: ${{ matrix.python-version }}
|
|
20
|
+
- run: python -m pip install ".[schema]"
|
|
21
|
+
- run: python -m unittest discover -s tests
|
|
22
|
+
- run: python -m repro_evidence_kit evidence validate examples/evidence-bundle.yaml
|
|
23
|
+
- run: python -m repro_evidence_kit manifest create examples/dummy-binary -o /tmp/repro-manifest.json
|
|
24
|
+
- run: python scripts/smoke_examples.py
|
|
25
|
+
- name: Leakage audit
|
|
26
|
+
run: |
|
|
27
|
+
! grep -RInE 'PRO7|조조전|ekd5|EEX|S_44|R_44|Phase4|Frida|xdbg|Cao Cao|save editor|game mod|/mnt/c/Users/xodnr/Desktop/new|_modding_tools' . --exclude-dir=.git --exclude=ci.yml
|
|
28
|
+
|
|
29
|
+
package:
|
|
30
|
+
name: Build package distributions
|
|
31
|
+
runs-on: ubuntu-latest
|
|
32
|
+
steps:
|
|
33
|
+
- uses: actions/checkout@v4
|
|
34
|
+
- uses: actions/setup-python@v5
|
|
35
|
+
with:
|
|
36
|
+
python-version: '3.12'
|
|
37
|
+
- run: python -m pip install build twine
|
|
38
|
+
- run: python -m build
|
|
39
|
+
- run: python scripts/check_dist.py
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
name: Publish to PyPI
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
release:
|
|
5
|
+
types: [published]
|
|
6
|
+
|
|
7
|
+
permissions:
|
|
8
|
+
contents: read
|
|
9
|
+
|
|
10
|
+
jobs:
|
|
11
|
+
build:
|
|
12
|
+
name: Build release distributions
|
|
13
|
+
runs-on: ubuntu-latest
|
|
14
|
+
steps:
|
|
15
|
+
- uses: actions/checkout@v4
|
|
16
|
+
- uses: actions/setup-python@v5
|
|
17
|
+
with:
|
|
18
|
+
python-version: '3.12'
|
|
19
|
+
- name: Confirm tag matches package version
|
|
20
|
+
run: |
|
|
21
|
+
package_version="$(python -c 'import tomllib; print(tomllib.load(open("pyproject.toml", "rb"))["project"]["version"])')"
|
|
22
|
+
test "$GITHUB_REF_NAME" = "v$package_version"
|
|
23
|
+
- run: python -m pip install build twine
|
|
24
|
+
- run: python -m build
|
|
25
|
+
- run: python scripts/check_dist.py
|
|
26
|
+
- uses: actions/upload-artifact@v4
|
|
27
|
+
with:
|
|
28
|
+
name: python-package-distributions
|
|
29
|
+
path: dist/
|
|
30
|
+
if-no-files-found: error
|
|
31
|
+
|
|
32
|
+
publish:
|
|
33
|
+
name: Publish release to PyPI
|
|
34
|
+
needs: build
|
|
35
|
+
runs-on: ubuntu-latest
|
|
36
|
+
environment:
|
|
37
|
+
name: pypi
|
|
38
|
+
url: https://pypi.org/p/repro-evidence-kit
|
|
39
|
+
permissions:
|
|
40
|
+
id-token: write
|
|
41
|
+
steps:
|
|
42
|
+
- uses: actions/download-artifact@v4
|
|
43
|
+
with:
|
|
44
|
+
name: python-package-distributions
|
|
45
|
+
path: dist/
|
|
46
|
+
- uses: pypa/gh-action-pypi-publish@release/v1
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## Unreleased
|
|
4
|
+
|
|
5
|
+
## 0.4.1 - 2026-06-07
|
|
6
|
+
|
|
7
|
+
- Add a Python 3.10, 3.11, and 3.12 CI matrix using non-editable package installs.
|
|
8
|
+
- Add checked wheel/sdist builds and a release-triggered PyPI Trusted Publishing workflow.
|
|
9
|
+
|
|
10
|
+
## 0.4.0 - 2026-06-06
|
|
11
|
+
|
|
12
|
+
- Add signature sidecar JSON Schema validation and packaged-schema regression coverage.
|
|
13
|
+
- Improve `evidence verify-signature` with text output, structured error details, and optional sidecar schema checks.
|
|
14
|
+
- Add `evidence sign --dry-run` for non-writing sidecar previews.
|
|
15
|
+
- Add sandbox required-change checks, SARIF output, and JUnit output for evidence validation.
|
|
16
|
+
- Add synthetic signed-bundle examples, use-case docs, release checklist, example smoke script, and release/install smoke script.
|
|
17
|
+
|
|
18
|
+
## 0.3.0 - 2026-06-05
|
|
19
|
+
|
|
20
|
+
- Add `evidence sign` and `evidence verify-signature` for local `hmac-sha256` evidence-bundle sidecars.
|
|
21
|
+
- Document signed-bundle boundaries: local tamper detection only, unsigned bundles remain supported, and signatures do not prove command execution or artifact semantics.
|
|
22
|
+
- Add regression coverage for successful signature verification and payload-mismatch failures.
|
|
23
|
+
|
|
24
|
+
## 0.2.0 - 2026-06-02
|
|
25
|
+
|
|
26
|
+
- Add JUnit XML output for `verify sandbox-run` CI report consumers.
|
|
27
|
+
- Add a signed evidence bundles design note that defines the sidecar-first support boundary.
|
|
28
|
+
|
|
29
|
+
## 0.1.2 - 2026-06-02
|
|
30
|
+
|
|
31
|
+
- Add optional JSON Schema validation for evidence bundles.
|
|
32
|
+
- Add include/exclude filters for manifest creation.
|
|
33
|
+
- Add Markdown output for `manifest diff` reports.
|
|
34
|
+
- Document stable CLI exit-code meanings and add regression coverage.
|
|
35
|
+
- Expand GitHub Actions cookbook coverage for schema-backed filtered manifest workflows.
|
|
36
|
+
- Clarify source-based installation until a package index release exists.
|
|
37
|
+
|
|
38
|
+
## 0.1.1 - 2026-05-31
|
|
39
|
+
|
|
40
|
+
- Add Windows-style path separator normalization for manifest diff and sandbox verification.
|
|
41
|
+
- Expand README maintainer positioning.
|
|
42
|
+
|
|
43
|
+
## 0.1.0 - 2026-05-30
|
|
44
|
+
|
|
45
|
+
- Initial release candidate.
|
|
46
|
+
- Add manifest creation and diffing.
|
|
47
|
+
- Add sandbox output verification.
|
|
48
|
+
- Add evidence bundle validation.
|
|
49
|
+
- Add synthetic examples and documentation.
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# Contributing
|
|
2
|
+
|
|
3
|
+
Contributions are welcome if they keep the project general-purpose and reviewable.
|
|
4
|
+
|
|
5
|
+
Do not submit proprietary binaries, copyrighted samples, private case data, credentials, or target-specific reverse-engineering artifacts. Use synthetic fixtures generated from source code whenever possible.
|
|
6
|
+
|
|
7
|
+
Before opening a pull request, run:
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
python -m unittest discover -s tests
|
|
11
|
+
python -m repro_evidence_kit evidence validate examples/evidence-bundle.yaml
|
|
12
|
+
```
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
Apache License
|
|
2
|
+
Version 2.0, January 2004
|
|
3
|
+
https://www.apache.org/licenses/
|
|
4
|
+
|
|
5
|
+
Copyright 2026 xodnr927-byte
|
|
6
|
+
|
|
7
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
8
|
+
you may not use this file except in compliance with the License.
|
|
9
|
+
You may obtain a copy of the License at
|
|
10
|
+
|
|
11
|
+
https://www.apache.org/licenses/LICENSE-2.0
|
|
12
|
+
|
|
13
|
+
Unless required by applicable law or agreed to in writing, software
|
|
14
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
15
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
16
|
+
See the License for the specific language governing permissions and
|
|
17
|
+
limitations under the License.
|
|
@@ -0,0 +1,153 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: repro-evidence-kit
|
|
3
|
+
Version: 0.4.1
|
|
4
|
+
Summary: Reproducible artifact manifests, sandbox-output verification, and evidence-bundle validation.
|
|
5
|
+
Project-URL: Homepage, https://github.com/xodnr927-byte/repro-evidence-kit
|
|
6
|
+
Project-URL: Issues, https://github.com/xodnr927-byte/repro-evidence-kit/issues
|
|
7
|
+
Author: xodnr927-byte
|
|
8
|
+
License-Expression: Apache-2.0
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Keywords: artifact,binary-analysis,evidence,manifest,reproducibility,sha256
|
|
11
|
+
Classifier: Development Status :: 3 - Alpha
|
|
12
|
+
Classifier: Environment :: Console
|
|
13
|
+
Classifier: Intended Audience :: Developers
|
|
14
|
+
Classifier: License :: OSI Approved :: Apache Software License
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
19
|
+
Classifier: Topic :: Security
|
|
20
|
+
Classifier: Topic :: Software Development :: Quality Assurance
|
|
21
|
+
Requires-Python: >=3.10
|
|
22
|
+
Requires-Dist: pyyaml>=6.0
|
|
23
|
+
Provides-Extra: dev
|
|
24
|
+
Requires-Dist: jsonschema>=4.0; extra == 'dev'
|
|
25
|
+
Requires-Dist: pytest>=8.0; extra == 'dev'
|
|
26
|
+
Provides-Extra: schema
|
|
27
|
+
Requires-Dist: jsonschema>=4.0; extra == 'schema'
|
|
28
|
+
Description-Content-Type: text/markdown
|
|
29
|
+
|
|
30
|
+
# repro-evidence-kit
|
|
31
|
+
|
|
32
|
+
[](https://github.com/xodnr927-byte/repro-evidence-kit/actions/workflows/ci.yml)
|
|
33
|
+
[](https://github.com/xodnr927-byte/repro-evidence-kit/releases)
|
|
34
|
+
[](https://www.python.org/)
|
|
35
|
+
[](LICENSE)
|
|
36
|
+
|
|
37
|
+
A small command-line toolkit for reproducible artifact verification in binary analysis, security research, and automation workflows.
|
|
38
|
+
|
|
39
|
+
It creates hash manifests, compares experiment outputs, and validates evidence bundles so results can be reviewed without relying on private source data.
|
|
40
|
+
|
|
41
|
+
## Why this matters
|
|
42
|
+
|
|
43
|
+
Generated artifacts are hard to review when the only proof is a large log, a private input tree, or a verbal claim that "nothing important changed." `repro-evidence-kit` keeps the review surface small: it records byte hashes, separates expected output changes from unexpected ones, and stores enough command context for another maintainer to rerun or challenge the evidence.
|
|
44
|
+
|
|
45
|
+
The project is intentionally target-neutral. It should help maintainers in CI, security research, binary-analysis, data-processing, and automation workflows without requiring them to publish proprietary samples or project-specific case files.
|
|
46
|
+
|
|
47
|
+
## Use cases
|
|
48
|
+
|
|
49
|
+
- Review what changed during artifact-heavy CI or release automation.
|
|
50
|
+
- Verify that a sandboxed experiment only produced explicitly allowed outputs.
|
|
51
|
+
- Attach compact, hash-backed evidence bundles to pull requests or research notes.
|
|
52
|
+
- Keep generated reports reviewable without publishing private input data.
|
|
53
|
+
|
|
54
|
+
## What this proves
|
|
55
|
+
|
|
56
|
+
- File manifests prove byte identity for the files they list.
|
|
57
|
+
- Manifest diffs separate expected artifact changes from unexpected ones.
|
|
58
|
+
- Sandbox verification proves the observed output set stayed inside an explicit allowlist.
|
|
59
|
+
- Evidence bundles preserve command context, inputs, outputs, and hashes for review.
|
|
60
|
+
- Signed sidecars add local tamper detection for exact bundle bytes.
|
|
61
|
+
|
|
62
|
+
## What this does not prove
|
|
63
|
+
|
|
64
|
+
- Hashes do not prove that generated outputs are semantically correct.
|
|
65
|
+
- A passing sandbox check does not prove that a command was safe.
|
|
66
|
+
- Signed sidecars do not prove signer identity, key trust, command execution, or artifact semantics.
|
|
67
|
+
- Private or proprietary inputs still require reviewer judgment outside this repository.
|
|
68
|
+
|
|
69
|
+
## Features
|
|
70
|
+
|
|
71
|
+
- Create deterministic SHA-256 manifests for files or filtered directory trees.
|
|
72
|
+
- Diff two manifests to identify added, removed, changed, and unchanged artifacts.
|
|
73
|
+
- Verify sandbox/experiment outputs against explicit allowlists, with optional JUnit XML for CI report consumers.
|
|
74
|
+
- Validate simple YAML or JSON evidence bundles, with optional JSON Schema checks.
|
|
75
|
+
- Sign and verify evidence bundle sidecars with a local-key tamper-detection prototype.
|
|
76
|
+
- Includes only synthetic public examples.
|
|
77
|
+
|
|
78
|
+
## Install
|
|
79
|
+
|
|
80
|
+
Until a package index release is published, install from the repository:
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
pip install "git+https://github.com/xodnr927-byte/repro-evidence-kit.git@v0.4.1"
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
For local development:
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
python -m venv .venv
|
|
90
|
+
. .venv/bin/activate
|
|
91
|
+
pip install -e .
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
## Quick start
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
repro-evidence manifest create examples/dummy-binary -o before.json
|
|
98
|
+
repro-evidence manifest diff before.json before.json
|
|
99
|
+
repro-evidence evidence validate examples/evidence-bundle.yaml
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
For larger artifact trees, filter manifests with explicit include/exclude patterns:
|
|
103
|
+
|
|
104
|
+
```bash
|
|
105
|
+
repro-evidence manifest create artifacts --include reports --exclude "*.tmp" -o manifest.json
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
For stricter evidence-bundle checks, install the optional schema extra and validate against the checked-in JSON Schema:
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
pip install "repro-evidence-kit[schema] @ git+https://github.com/xodnr927-byte/repro-evidence-kit.git@v0.4.1"
|
|
112
|
+
repro-evidence evidence validate examples/evidence-bundle.yaml --schema
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
|
|
116
|
+
Signed bundle sidecars are optional. For a local tamper-detection prototype, create or provide local trust material and keep it out of git:
|
|
117
|
+
|
|
118
|
+
```bash
|
|
119
|
+
printf 'synthetic local test key only\n' > local-test.key
|
|
120
|
+
repro-evidence evidence sign examples/evidence-bundle.yaml --key local-test.key -o evidence-bundle.yaml.sig.json
|
|
121
|
+
repro-evidence evidence verify-signature examples/evidence-bundle.yaml --signature evidence-bundle.yaml.sig.json --key local-test.key
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Sandbox verification compares a baseline manifest with an after-run manifest:
|
|
125
|
+
|
|
126
|
+
```bash
|
|
127
|
+
repro-evidence verify sandbox-run before.json after.json --allow-added report.json
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
The command exits `0` when all changes are allowed and `1` when unexpected changes are present.
|
|
131
|
+
|
|
132
|
+
## Documentation
|
|
133
|
+
|
|
134
|
+
- [CLI reference](docs/cli.md)
|
|
135
|
+
- [CLI exit codes](docs/cli-exit-codes.md)
|
|
136
|
+
- [Tutorial](docs/tutorial.md)
|
|
137
|
+
- [Evidence bundle format](docs/evidence-bundle-format.md)
|
|
138
|
+
- [Use cases](docs/use-cases.md)
|
|
139
|
+
- [Signed evidence bundles design note](docs/signed-bundles.md)
|
|
140
|
+
- [Maintainer workflow](docs/maintainer-workflow.md)
|
|
141
|
+
- [Release checklist](docs/release-checklist.md)
|
|
142
|
+
- [PyPI publishing](docs/publishing.md)
|
|
143
|
+
- [GitHub Actions cookbook](docs/github-actions.md) — CI recipes for validation, manifests, sandbox checks, and schema-backed filtered workflows.
|
|
144
|
+
- [Design principles](docs/design-principles.md)
|
|
145
|
+
- [Roadmap](ROADMAP.md)
|
|
146
|
+
|
|
147
|
+
## Data policy
|
|
148
|
+
|
|
149
|
+
This repository is for generic reproducibility tooling. Do not add proprietary binaries, private datasets, copyrighted samples, live credentials, forensic case data, or project-specific reverse-engineering targets. Public examples must be synthetic or clearly redistributable.
|
|
150
|
+
|
|
151
|
+
## Status
|
|
152
|
+
|
|
153
|
+
`0.4.x` is an early maintainer-tooling release series. The CLI and schema stay intentionally small, conservative, and synthetic-example-only.
|
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
# repro-evidence-kit
|
|
2
|
+
|
|
3
|
+
[](https://github.com/xodnr927-byte/repro-evidence-kit/actions/workflows/ci.yml)
|
|
4
|
+
[](https://github.com/xodnr927-byte/repro-evidence-kit/releases)
|
|
5
|
+
[](https://www.python.org/)
|
|
6
|
+
[](LICENSE)
|
|
7
|
+
|
|
8
|
+
A small command-line toolkit for reproducible artifact verification in binary analysis, security research, and automation workflows.
|
|
9
|
+
|
|
10
|
+
It creates hash manifests, compares experiment outputs, and validates evidence bundles so results can be reviewed without relying on private source data.
|
|
11
|
+
|
|
12
|
+
## Why this matters
|
|
13
|
+
|
|
14
|
+
Generated artifacts are hard to review when the only proof is a large log, a private input tree, or a verbal claim that "nothing important changed." `repro-evidence-kit` keeps the review surface small: it records byte hashes, separates expected output changes from unexpected ones, and stores enough command context for another maintainer to rerun or challenge the evidence.
|
|
15
|
+
|
|
16
|
+
The project is intentionally target-neutral. It should help maintainers in CI, security research, binary-analysis, data-processing, and automation workflows without requiring them to publish proprietary samples or project-specific case files.
|
|
17
|
+
|
|
18
|
+
## Use cases
|
|
19
|
+
|
|
20
|
+
- Review what changed during artifact-heavy CI or release automation.
|
|
21
|
+
- Verify that a sandboxed experiment only produced explicitly allowed outputs.
|
|
22
|
+
- Attach compact, hash-backed evidence bundles to pull requests or research notes.
|
|
23
|
+
- Keep generated reports reviewable without publishing private input data.
|
|
24
|
+
|
|
25
|
+
## What this proves
|
|
26
|
+
|
|
27
|
+
- File manifests prove byte identity for the files they list.
|
|
28
|
+
- Manifest diffs separate expected artifact changes from unexpected ones.
|
|
29
|
+
- Sandbox verification proves the observed output set stayed inside an explicit allowlist.
|
|
30
|
+
- Evidence bundles preserve command context, inputs, outputs, and hashes for review.
|
|
31
|
+
- Signed sidecars add local tamper detection for exact bundle bytes.
|
|
32
|
+
|
|
33
|
+
## What this does not prove
|
|
34
|
+
|
|
35
|
+
- Hashes do not prove that generated outputs are semantically correct.
|
|
36
|
+
- A passing sandbox check does not prove that a command was safe.
|
|
37
|
+
- Signed sidecars do not prove signer identity, key trust, command execution, or artifact semantics.
|
|
38
|
+
- Private or proprietary inputs still require reviewer judgment outside this repository.
|
|
39
|
+
|
|
40
|
+
## Features
|
|
41
|
+
|
|
42
|
+
- Create deterministic SHA-256 manifests for files or filtered directory trees.
|
|
43
|
+
- Diff two manifests to identify added, removed, changed, and unchanged artifacts.
|
|
44
|
+
- Verify sandbox/experiment outputs against explicit allowlists, with optional JUnit XML for CI report consumers.
|
|
45
|
+
- Validate simple YAML or JSON evidence bundles, with optional JSON Schema checks.
|
|
46
|
+
- Sign and verify evidence bundle sidecars with a local-key tamper-detection prototype.
|
|
47
|
+
- Includes only synthetic public examples.
|
|
48
|
+
|
|
49
|
+
## Install
|
|
50
|
+
|
|
51
|
+
Until a package index release is published, install from the repository:
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
pip install "git+https://github.com/xodnr927-byte/repro-evidence-kit.git@v0.4.1"
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
For local development:
|
|
58
|
+
|
|
59
|
+
```bash
|
|
60
|
+
python -m venv .venv
|
|
61
|
+
. .venv/bin/activate
|
|
62
|
+
pip install -e .
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
## Quick start
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
repro-evidence manifest create examples/dummy-binary -o before.json
|
|
69
|
+
repro-evidence manifest diff before.json before.json
|
|
70
|
+
repro-evidence evidence validate examples/evidence-bundle.yaml
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
For larger artifact trees, filter manifests with explicit include/exclude patterns:
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
repro-evidence manifest create artifacts --include reports --exclude "*.tmp" -o manifest.json
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
For stricter evidence-bundle checks, install the optional schema extra and validate against the checked-in JSON Schema:
|
|
80
|
+
|
|
81
|
+
```bash
|
|
82
|
+
pip install "repro-evidence-kit[schema] @ git+https://github.com/xodnr927-byte/repro-evidence-kit.git@v0.4.1"
|
|
83
|
+
repro-evidence evidence validate examples/evidence-bundle.yaml --schema
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
|
|
87
|
+
Signed bundle sidecars are optional. For a local tamper-detection prototype, create or provide local trust material and keep it out of git:
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
printf 'synthetic local test key only\n' > local-test.key
|
|
91
|
+
repro-evidence evidence sign examples/evidence-bundle.yaml --key local-test.key -o evidence-bundle.yaml.sig.json
|
|
92
|
+
repro-evidence evidence verify-signature examples/evidence-bundle.yaml --signature evidence-bundle.yaml.sig.json --key local-test.key
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
Sandbox verification compares a baseline manifest with an after-run manifest:
|
|
96
|
+
|
|
97
|
+
```bash
|
|
98
|
+
repro-evidence verify sandbox-run before.json after.json --allow-added report.json
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
The command exits `0` when all changes are allowed and `1` when unexpected changes are present.
|
|
102
|
+
|
|
103
|
+
## Documentation
|
|
104
|
+
|
|
105
|
+
- [CLI reference](docs/cli.md)
|
|
106
|
+
- [CLI exit codes](docs/cli-exit-codes.md)
|
|
107
|
+
- [Tutorial](docs/tutorial.md)
|
|
108
|
+
- [Evidence bundle format](docs/evidence-bundle-format.md)
|
|
109
|
+
- [Use cases](docs/use-cases.md)
|
|
110
|
+
- [Signed evidence bundles design note](docs/signed-bundles.md)
|
|
111
|
+
- [Maintainer workflow](docs/maintainer-workflow.md)
|
|
112
|
+
- [Release checklist](docs/release-checklist.md)
|
|
113
|
+
- [PyPI publishing](docs/publishing.md)
|
|
114
|
+
- [GitHub Actions cookbook](docs/github-actions.md) — CI recipes for validation, manifests, sandbox checks, and schema-backed filtered workflows.
|
|
115
|
+
- [Design principles](docs/design-principles.md)
|
|
116
|
+
- [Roadmap](ROADMAP.md)
|
|
117
|
+
|
|
118
|
+
## Data policy
|
|
119
|
+
|
|
120
|
+
This repository is for generic reproducibility tooling. Do not add proprietary binaries, private datasets, copyrighted samples, live credentials, forensic case data, or project-specific reverse-engineering targets. Public examples must be synthetic or clearly redistributable.
|
|
121
|
+
|
|
122
|
+
## Status
|
|
123
|
+
|
|
124
|
+
`0.4.x` is an early maintainer-tooling release series. The CLI and schema stay intentionally small, conservative, and synthetic-example-only.
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
# Roadmap
|
|
2
|
+
|
|
3
|
+
## Recently completed
|
|
4
|
+
|
|
5
|
+
These are available in `v0.4.0`:
|
|
6
|
+
|
|
7
|
+
- Markdown output for `manifest diff` reports.
|
|
8
|
+
- Include/exclude filters for large artifact-tree manifests.
|
|
9
|
+
- Optional JSON Schema validation for evidence bundles.
|
|
10
|
+
- Stable CLI exit-code documentation and regression coverage.
|
|
11
|
+
- GitHub Actions cookbook and CI leakage audit.
|
|
12
|
+
- Minimal signed evidence bundle sidecar prototype.
|
|
13
|
+
|
|
14
|
+
- Signature sidecar JSON Schema and packaged schema regression tests.
|
|
15
|
+
- Reviewer-friendly `verify-signature --format text` output and structured JSON error details.
|
|
16
|
+
- `evidence sign --dry-run` for non-writing sidecar preview.
|
|
17
|
+
- Synthetic signed-bundle example, release checklist, use-cases page, and example smoke script.
|
|
18
|
+
- Sandbox SARIF output, required-change checks, and evidence-validation JUnit output.
|
|
19
|
+
- Fresh-environment release/install smoke script for tagged source references.
|
|
20
|
+
|
|
21
|
+
## Near-term polish
|
|
22
|
+
|
|
23
|
+
- Keep examples synthetic-only.
|
|
24
|
+
- Expand CI recipes for signed-bundle verification and example smoke checks.
|
|
25
|
+
- Complete the first PyPI release after Trusted Publisher configuration.
|
|
26
|
+
- Add SARIF or additional JUnit adapters only after the JSON/text contracts stay stable.
|
|
27
|
+
|
|
28
|
+
## Later ideas
|
|
29
|
+
|
|
30
|
+
- SARIF output for CI code-scanning integrations.
|
|
31
|
+
- Richer signed bundle trust policies after the sidecar contract is stable.
|
|
32
|
+
|
|
33
|
+
## Non-goals
|
|
34
|
+
|
|
35
|
+
- Storing private source data.
|
|
36
|
+
- Shipping proprietary binary samples.
|
|
37
|
+
- Becoming a target-specific reverse-engineering framework.
|
|
38
|
+
- Claiming semantic correctness from hashes alone.
|
|
@@ -0,0 +1,5 @@
|
|
|
1
|
+
# Security policy
|
|
2
|
+
|
|
3
|
+
Please do not attach private binaries, credentials, forensic datasets, or sensitive logs to public issues.
|
|
4
|
+
|
|
5
|
+
If you believe the toolkit mishandles paths, hashes, evidence bundles, or generated reports in a way that could disclose sensitive data, open a private security advisory or contact the maintainer through GitHub.
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
# CLI exit codes
|
|
2
|
+
|
|
3
|
+
All commands use the same top-level exit-code contract.
|
|
4
|
+
|
|
5
|
+
| Code | Meaning | Examples |
|
|
6
|
+
| --- | --- | --- |
|
|
7
|
+
| `0` | The command completed and the checked predicate passed. | Manifest written, manifests diffed, evidence bundle valid, signature verified. |
|
|
8
|
+
| `1` | The command completed and found an expected validation or verification failure. | Invalid evidence bundle, unexpected sandbox outputs, signature mismatch, unsupported signature sidecar metadata. |
|
|
9
|
+
| `2` | The command could not complete because of an input, parsing, filesystem, dependency, or runtime error. | Missing JSON file, malformed sidecar JSON, unreadable evidence file, schema validation without `jsonschema`. |
|
|
10
|
+
|
|
11
|
+
Use `1` as a CI policy failure and `2` as an infrastructure or invocation failure.
|