twincheck 0.12.0a1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (104) hide show
  1. twincheck-0.12.0a1/CHANGELOG.md +142 -0
  2. twincheck-0.12.0a1/CODE_OF_CONDUCT.md +43 -0
  3. twincheck-0.12.0a1/CONTRIBUTING.md +42 -0
  4. twincheck-0.12.0a1/LICENSE +21 -0
  5. twincheck-0.12.0a1/MANIFEST.in +15 -0
  6. twincheck-0.12.0a1/PKG-INFO +267 -0
  7. twincheck-0.12.0a1/README.md +238 -0
  8. twincheck-0.12.0a1/SECURITY.md +25 -0
  9. twincheck-0.12.0a1/benchmarks/README.md +70 -0
  10. twincheck-0.12.0a1/benchmarks/compare_exploration_modes.py +165 -0
  11. twincheck-0.12.0a1/benchmarks/exploration-corpus-v1/contracts/cart-compatibility-v1.json +15 -0
  12. twincheck-0.12.0a1/benchmarks/exploration-corpus-v1/manifest.json +135 -0
  13. twincheck-0.12.0a1/benchmarks/exploration-corpus-v1/sequences/cart-create-add-read-twice-v1.json +36 -0
  14. twincheck-0.12.0a1/benchmarks/exploration-corpus-v1/sequences/cart-create-add-read-v1.json +31 -0
  15. twincheck-0.12.0a1/benchmarks/exploration-corpus-v1/sequences/reset-v1.json +22 -0
  16. twincheck-0.12.0a1/benchmarks/exploration-corpus-v1/service.py +120 -0
  17. twincheck-0.12.0a1/benchmarks/exploration-corpus-v1.schema.json +110 -0
  18. twincheck-0.12.0a1/benchmarks/exploration-corpus-v2/contracts/booking-compatibility-v1.json +8 -0
  19. twincheck-0.12.0a1/benchmarks/exploration-corpus-v2/manifest.json +70 -0
  20. twincheck-0.12.0a1/benchmarks/exploration-corpus-v2/sequences/booking-session-reserve-read-v1.json +9 -0
  21. twincheck-0.12.0a1/benchmarks/exploration-corpus-v2/sequences/reset-v1.json +5 -0
  22. twincheck-0.12.0a1/benchmarks/exploration-corpus-v2/service.py +73 -0
  23. twincheck-0.12.0a1/benchmarks/exploration-corpus-v2.schema.json +18 -0
  24. twincheck-0.12.0a1/benchmarks/run_exploration_protocol.py +445 -0
  25. twincheck-0.12.0a1/benchmarks/run_multi_domain_protocol.py +139 -0
  26. twincheck-0.12.0a1/benchmarks/run_smoke.py +67 -0
  27. twincheck-0.12.0a1/benchmarks/score_exploration_protocol.py +130 -0
  28. twincheck-0.12.0a1/benchmarks/stateful-corpus-v1/README.md +59 -0
  29. twincheck-0.12.0a1/benchmarks/stateful-corpus-v1/contract.json +15 -0
  30. twincheck-0.12.0a1/benchmarks/stateful-corpus-v1/reset-sequence.json +22 -0
  31. twincheck-0.12.0a1/benchmarks/stateful-corpus-v1/sequence.json +37 -0
  32. twincheck-0.12.0a1/benchmarks/stateful-corpus-v2/cases/cart-read-twice.json +36 -0
  33. twincheck-0.12.0a1/benchmarks/stateful-corpus-v2/cases/cart-value-drift.json +37 -0
  34. twincheck-0.12.0a1/benchmarks/stateful-corpus-v2/cases/idempotency-retry.json +27 -0
  35. twincheck-0.12.0a1/benchmarks/stateful-corpus-v2/cases/volatile-control.json +14 -0
  36. twincheck-0.12.0a1/benchmarks/stateful-corpus-v2/contracts/ignore-volatile-id.json +15 -0
  37. twincheck-0.12.0a1/benchmarks/stateful-corpus-v2/manifest.json +138 -0
  38. twincheck-0.12.0a1/benchmarks/stateful-corpus-v2/reset-sequence.json +26 -0
  39. twincheck-0.12.0a1/benchmarks/stateful-corpus-v2/service.py +158 -0
  40. twincheck-0.12.0a1/benchmarks/stateful-corpus-v2.schema.json +100 -0
  41. twincheck-0.12.0a1/benchmarks/verify_stateful_corpus.py +453 -0
  42. twincheck-0.12.0a1/docs/adaptive-stateful-exploration-v1.md +58 -0
  43. twincheck-0.12.0a1/docs/architecture.md +49 -0
  44. twincheck-0.12.0a1/docs/compatibility-contract-v1.schema.json +74 -0
  45. twincheck-0.12.0a1/docs/competitive-landscape.md +39 -0
  46. twincheck-0.12.0a1/docs/counterexample-pack-v1.schema.json +93 -0
  47. twincheck-0.12.0a1/docs/design-decisions.md +120 -0
  48. twincheck-0.12.0a1/docs/project-plan.md +243 -0
  49. twincheck-0.12.0a1/docs/publication-safety-audit.md +88 -0
  50. twincheck-0.12.0a1/docs/python-api.md +105 -0
  51. twincheck-0.12.0a1/docs/quickstart.md +169 -0
  52. twincheck-0.12.0a1/docs/release-process.md +70 -0
  53. twincheck-0.12.0a1/docs/report-schema-v1.schema.json +547 -0
  54. twincheck-0.12.0a1/docs/roadmap.md +142 -0
  55. twincheck-0.12.0a1/docs/schema-generation.md +35 -0
  56. twincheck-0.12.0a1/docs/stateful-http-sequence-v1.schema.json +62 -0
  57. twincheck-0.12.0a1/docs/stateful-regression-pack-v1.md +26 -0
  58. twincheck-0.12.0a1/docs/stateful-regression-pack-v1.schema.json +145 -0
  59. twincheck-0.12.0a1/docs/threat-model.md +42 -0
  60. twincheck-0.12.0a1/docs/v0.7-verification.md +10 -0
  61. twincheck-0.12.0a1/examples/README.md +58 -0
  62. twincheck-0.12.0a1/examples/contracts/http-api-v1.json +11 -0
  63. twincheck-0.12.0a1/examples/contracts/json-api-v1.json +13 -0
  64. twincheck-0.12.0a1/examples/contracts/stateful-cart-v1.json +11 -0
  65. twincheck-0.12.0a1/examples/schemas/quote-request-v1.json +22 -0
  66. twincheck-0.12.0a1/examples/sequences/cart-create-add-read-v1.json +37 -0
  67. twincheck-0.12.0a1/examples/sequences/cart-reset-v1.json +22 -0
  68. twincheck-0.12.0a1/pyproject.toml +48 -0
  69. twincheck-0.12.0a1/scripts/verify_distribution.py +206 -0
  70. twincheck-0.12.0a1/setup.cfg +4 -0
  71. twincheck-0.12.0a1/src/twincheck/__init__.py +30 -0
  72. twincheck-0.12.0a1/src/twincheck/__main__.py +4 -0
  73. twincheck-0.12.0a1/src/twincheck/api.py +86 -0
  74. twincheck-0.12.0a1/src/twincheck/cli.py +557 -0
  75. twincheck-0.12.0a1/src/twincheck/compare.py +182 -0
  76. twincheck-0.12.0a1/src/twincheck/contract.py +232 -0
  77. twincheck-0.12.0a1/src/twincheck/engine.py +232 -0
  78. twincheck-0.12.0a1/src/twincheck/generators.py +143 -0
  79. twincheck-0.12.0a1/src/twincheck/http_engine.py +349 -0
  80. twincheck-0.12.0a1/src/twincheck/packs.py +197 -0
  81. twincheck-0.12.0a1/src/twincheck/report.py +101 -0
  82. twincheck-0.12.0a1/src/twincheck/runner.py +177 -0
  83. twincheck-0.12.0a1/src/twincheck/schema.py +657 -0
  84. twincheck-0.12.0a1/src/twincheck/stateful_explore.py +1153 -0
  85. twincheck-0.12.0a1/src/twincheck/stateful_http.py +975 -0
  86. twincheck-0.12.0a1/src/twincheck/stateful_packs.py +184 -0
  87. twincheck-0.12.0a1/src/twincheck.egg-info/PKG-INFO +267 -0
  88. twincheck-0.12.0a1/src/twincheck.egg-info/SOURCES.txt +102 -0
  89. twincheck-0.12.0a1/src/twincheck.egg-info/dependency_links.txt +1 -0
  90. twincheck-0.12.0a1/src/twincheck.egg-info/entry_points.txt +2 -0
  91. twincheck-0.12.0a1/src/twincheck.egg-info/requires.txt +3 -0
  92. twincheck-0.12.0a1/src/twincheck.egg-info/top_level.txt +1 -0
  93. twincheck-0.12.0a1/tests/test_benchmark_scorecard.py +49 -0
  94. twincheck-0.12.0a1/tests/test_contracts.py +92 -0
  95. twincheck-0.12.0a1/tests/test_engine_cli.py +183 -0
  96. twincheck-0.12.0a1/tests/test_generators.py +36 -0
  97. twincheck-0.12.0a1/tests/test_http_engine.py +227 -0
  98. twincheck-0.12.0a1/tests/test_packs.py +93 -0
  99. twincheck-0.12.0a1/tests/test_public_api.py +112 -0
  100. twincheck-0.12.0a1/tests/test_runner_compare.py +113 -0
  101. twincheck-0.12.0a1/tests/test_schema.py +80 -0
  102. twincheck-0.12.0a1/tests/test_stateful_corpus.py +81 -0
  103. twincheck-0.12.0a1/tests/test_stateful_http.py +801 -0
  104. twincheck-0.12.0a1/tests/test_stateful_packs.py +144 -0
@@ -0,0 +1,142 @@
1
+ # Changelog
2
+
3
+ ## Unreleased
4
+
5
+ No user-facing changes are currently unreleased.
6
+
7
+ ## 0.12.0a1 - Pinned exploration protocol corpus v1 and public alpha release
8
+
9
+ - Rewrite the README for public alpha users and add a standalone installation and quick-start guide.
10
+ - Add a publication-safety audit record and automated Gitleaks, CodeQL, OpenSSF Scorecard and Dependabot configuration.
11
+ - Add a draft GitHub Release workflow that builds and verifies artifacts from version tags.
12
+ - Add a PyPI Trusted Publishing workflow gated by the GitHub `pypi` environment and a published GitHub Release.
13
+ - Require the quick-start guide to be included in verified source distributions.
14
+ - Clarify the alpha release and compatibility policy.
15
+ - Pin GitHub Actions dependencies to immutable commit SHAs for supply-chain integrity.
16
+
17
+ - Add a separate, versioned synthetic exploration corpus with clean-control, response-value drift, stateful read-ordering, and paired-field interaction cases.
18
+ - Pin fixture, schema and protocol-runner SHA-256 digests and validate the corpus manifest and workflow/contract schemas before executing targets.
19
+ - Add a matched-budget static/adaptive protocol runner that checks expected findings, records sanitized schedule fingerprints, and repeats each fixed seed to test reproducibility.
20
+ - Report per-case outcomes, paired detection categories, schedule stability, run times, seeds and budgets while explicitly limiting claims to the synthetic fixtures.
21
+ - Run the protocol in the Linux/Windows CI matrix and include its files in the verified source archive.
22
+
23
+ This remains an alpha validation fixture, not an external benchmark or claim of general effectiveness. The paired-field interaction is an intentionally narrow toy defect and its seed is pinned to a known schedule.
24
+
25
+ ## 0.11.0a1 - Supported public Python API v1 (alpha)
26
+
27
+ - Expose a versioned Python API for command-line differential comparisons, compatibility contracts, and JSON/JUnit report output.
28
+ - Re-export the documented API symbols at the package root and define API_VERSION = 1 as the explicit public compatibility marker.
29
+ - Add API compatibility tests for exports, deterministic generated-case comparison, contract loading, serialization and report writers.
30
+ - Add a clean-wheel API smoke test and require the public API documentation and tests in the source archive.
31
+ - Document which Python imports are supported, how breaking API changes are versioned, and which HTTP workflows remain CLI-only.
32
+
33
+ This remains pre-1.0 software. API v1 is an explicit compatibility commitment for the documented exports, not a guarantee for internal modules, every report field, or every CLI option.
34
+
35
+ ## 0.10.0a1 - Observation-driven stateful exploration (alpha)
36
+
37
+ - Add an opt-in adaptive exploration mode that generates paired scalar-boundary candidates after newly observed response branches, adjacent transitions, or difference signatures.
38
+ - Preserve static seeded exploration as the default and record the algorithm version, generation rationale, selection rationale, candidate counts, budget and stop reason.
39
+ - Add a bounded wall-clock start gate alongside case, timeout, output and minimization-trial limits. Each reset-backed candidate/minimization workflow continues to use explicit reset and health semantics.
40
+ - Add an end-to-end test where adaptive exploration finds a two-field interaction defect that static exploration misses under the same two-case budget.
41
+ - Add a repeatable static-vs-adaptive comparison script for a pinned synthetic corpus fixture and run it in the Linux/Windows CI matrix.
42
+ - Document that the heuristic is not general state-machine learning and that response-shape novelty is not source-code coverage.
43
+ - Add distribution validation that builds a wheel and source archive, checks metadata, and installs the wheel in an isolated environment outside the checkout.
44
+ - Add a manual release-candidate workflow and an explicit pre-1.0 CLI/schema/Python API compatibility policy; the workflow does not publish to PyPI.
45
+ - Use SPDX-style PEP 639 license metadata and include the license file explicitly in built distributions.
46
+
47
+ This remains an alpha. The comparison is fixture-only and is not a general performance or superiority claim. An in-flight workflow can finish after the wall-clock deadline, bounded by the remaining per-request timeout and sequence limits.
48
+
49
+ ## 0.9.0a1 - Reproducible stateful evidence (alpha)
50
+
51
+ - Add a versioned five-case synthetic corpus: declared volatile-ID control, cart total drift, stateful read-ordering defect, idempotency/retry defect, and a reset-health guard that must block a stale target.
52
+ - Add `python benchmarks/verify_stateful_corpus.py` to check pinned fixture hashes, validate the corpus and project schemas, execute expected cases, inspect safe difference pointers, prove minimization preserves a named mismatch, export/replay a pack, and reject tampering.
53
+ - Add value-free per-step explanations with status/content-type metadata, the active comparison contract rules, and bounded safe JSON Pointer paths instead of duplicate field values.
54
+ - Expand CI to Linux/Python 3.11, 3.12 and 3.13 plus Windows/Python 3.11; upload synthetic corpus evidence as a GitHub Actions artifact.
55
+ - Keep schema validation in the optional `test` extra; the core library remains dependency-free.
56
+
57
+ This remains an alpha. The corpus is a deterministic correctness fixture, not a performance benchmark or an indication of coverage-guided testing. The synthetic API does not model distributed state, real databases, or concurrent payment processing.
58
+
59
+ ## 0.8.0a1 - Verifiable stateful regression workflows (alpha)
60
+
61
+ - Add an optional versioned reset-health contract to stateful sequences. After every configured reset, TwinCheck makes a separate read-only GET request and checks explicit scalar JSON Pointer assertions on both local targets before running the tested workflow.
62
+ - Record whether reset checking was status-only or assertion-verified. Health summaries retain assertion paths and pass/fail metadata, not observed response values.
63
+ - Add explicit `depends_on` metadata to stateful steps and reject dependencies that are duplicated, implicit, unknown, or ordered after their consumer.
64
+ - Prevent the bounded adjacent-swap explorer from deliberately reversing directly declared dependencies.
65
+ - Add a versioned synthetic stateful regression corpus, a broken-reset negative control, and reproducibility instructions.
66
+ - Add regression coverage for a reset endpoint that returns 200 without clearing state and for invalid dependency ordering.
67
+
68
+ This remains an alpha. A health contract verifies only the declared assertions, not all hidden state. Loopback-only HTTP targeting is not a sandbox. Instrumented coverage, dynamically generated mutation operators, and broad real-world benchmarking remain future work.
69
+
70
+ ## 0.7.0a1 - Portable stateful regression packs (alpha work in progress)
71
+
72
+ - Add `export-http-sequence-pack` to save one stateful finding as a regression pack containing the request sequence, optional reset workflow, and pinned compatibility contract.
73
+ - Add `replay-http-sequence-pack` to replay the pack against explicit loopback targets, with reset-before-repeat behavior and a required side-effect acknowledgement.
74
+ - Exclude response traces and request-header values from pack contents; provide authentication headers only at replay time.
75
+ - Verify file and canonical hashes before replay and refuse to overwrite existing export destinations.
76
+ - Add regression tests and a versioned pack-manifest schema.
77
+
78
+ This remains an alpha. A successful reset status cannot prove all hidden service state was cleared. Use disposable local services and synthetic data. GitHub publication and the complete Python-version CI matrix remain pending verification.
79
+
80
+ ## 0.6.0a1 - Feedback-guided stateful exploration
81
+
82
+ - Add value-free response-branch signatures using HTTP status, content type and JSON response shape, plus adjacent transition fingerprints.
83
+ - Execute the baseline first, then prioritize a seeded bounded candidate queue using uncovered mutation features, category balance and steps that recently exposed new response behavior.
84
+ - Record per-case priority scores/reasons, novel response branches/transitions, difference signatures and aggregate coverage counts in exploration reports.
85
+ - Extend minimization from step deletion to structural JSON shrinking (remove fields/items) and scalar simplification, with the same reset-backed failure-preservation check and a shared trial budget.
86
+ - Keep normal stateful comparison reports unchanged unless coverage capture is explicitly requested by the exploration engine; signatures do not store response values.
87
+ - Document the feedback model and its limits; bump alpha version to `0.6.0a1`.
88
+
89
+ This remains an alpha. The scheduler ranks a bounded pre-generated candidate pool; it does not synthesize arbitrary mutations from runtime traces. Response-shape fingerprints are not code coverage, and reset success still depends on the user's reset workflow actually clearing the relevant state.
90
+
91
+ ## 0.5.0a1 - Transition-aware stateful workflow exploration
92
+
93
+ - Add `explore-http-sequence` to generate reproducible JSON body boundary variants, adjacent step reorderings, step removals, and duplicate-step transitions.
94
+ - Require a user-authored reset workflow for every test variant and minimization trial; stop before the candidate workflow if either target's reset step fails or is not 2xx.
95
+ - Add bounded step-level delta debugging that preserves a selected mismatch category at the same named step.
96
+ - Store the mutation metadata, reset workflow definition and hashes, and minimized sequence so findings can be replayed with `replay-http-sequence`.
97
+ - Allow ordinary `compare-http-sequence` runs to opt into reset-before-repeat semantics.
98
+ - Extend test services, CLI validation, examples and regression coverage for deterministic exploration and faithful replay.
99
+
100
+ This is still an alpha. The reset endpoint is user-defined and trusted; a 2xx response cannot prove a service has fully reset. The mutator only explores the documented set of local mutations, and minimization removes steps rather than minimizing individual values. Run only against loopback test services using synthetic data.
101
+
102
+ ## 0.4.0a1 - Stateful HTTP compatibility workflows
103
+
104
+ - Add `compare-http-sequence` for bounded, user-authored multi-request workflows against two loopback HTTP implementations.
105
+ - Support response-value capture using JSON Pointer and safe interpolation into later request paths and JSON bodies.
106
+ - Preserve independent cookie jars within each target workflow and reset the jars between repeats; redirects remain disabled and proxy environment variables are ignored.
107
+ - Include source and canonical sequence hashes, source workflow definition, request-body hashes, per-step observations, and pinpointed differences in reports.
108
+ - Add `replay-http-sequence` for replaying a saved finding against explicitly supplied loopback targets; require `--allow-side-effects` and preserve the original compatibility contract.
109
+ - Add an example cart service, workflow and contract; add validation, capture, cookie, mismatch and replay tests.
110
+
111
+ This is an alpha milestone. The workflow is authored explicitly; TwinCheck does not automatically discover state transitions. Request bodies and response observations are part of failure evidence, so workflows must not contain real secrets or customer data. The adapter is not a sandbox.
112
+
113
+ ## 0.3.0a1 - Structured inputs and local HTTP adapter
114
+
115
+ - Add deterministic JSON Schema-guided request generation, including local JSON Pointer `$ref`, objects, arrays, enums/constants, common scalar constraints, and fail-closed diagnostics for unsupported validation keywords.
116
+ - Add `twincheck compare --schema` to drive existing command-line targets with schema-generated JSON inputs and record schema SHA-256 provenance.
117
+ - Add `twincheck compare-http` to compare two loopback HTTP implementations using schema-generated JSON request bodies.
118
+ - Compare HTTP status, JSON response bodies, and `content-type` by default; allow an explicit response-header allowlist and use the existing versioned compatibility contract for JSON Pointer ignores.
119
+ - Keep request header values out of reports, reject non-loopback HTTP targets, and add runnable quote-API examples.
120
+ - Add schema generator and HTTP adapter tests; no runtime dependencies added.
121
+
122
+ This is an alpha increment. JSON Schema generation implements a documented subset rather than the full specification. HTTP mode currently uses a fixed path and request JSON body; it is not a security sandbox or a general network scanner.
123
+
124
+ ## 0.2.0a1 - Development alpha
125
+
126
+ - Add repeated-run instability detection with per-run evidence for findings.
127
+ - Add bounded output retention using file-backed capture and output-limit reporting.
128
+ - Add `twincheck replay` to rerun a finding's exact input using explicitly supplied commands.
129
+ - Add JUnit XML export with one testcase per executed input.
130
+ - Add a versioned JSON Schema for evidence reports.
131
+ - Expand README, architecture, threat-model, design-decision and benchmark documentation.
132
+ - Make the default per-process timeout more tolerant for slower machines and CI.
133
+
134
+ This remains an alpha: the runner is not a sandbox, performance has not been benchmarked against mature fuzzing tools, and broad protocol/binary adapters are not implemented.
135
+
136
+ ## 0.1.0 - Initial prototype
137
+
138
+ - Add deterministic input generation for mixed, text, integer and JSON-like inputs.
139
+ - Add shell-free CLI execution with stdin and `{input}` argument support.
140
+ - Compare stdout, stderr, exit code, and timeout behavior.
141
+ - Add bounded delta-debugging input minimization.
142
+ - Add JSON reports, examples, standard-library tests, CI and project documentation.
@@ -0,0 +1,43 @@
1
+ # Code of Conduct
2
+
3
+ ## Our commitment
4
+
5
+ The TwinCheck community aims to be welcoming, respectful and constructive. We want participation to be safe and productive for people with different backgrounds, experience levels and perspectives.
6
+
7
+ ## Expected behavior
8
+
9
+ - Be respectful and considerate in discussions and reviews.
10
+ - Give and receive constructive feedback gracefully.
11
+ - Focus criticism on ideas, code and behavior rather than people.
12
+ - Acknowledge mistakes, learn from them and help repair their impact.
13
+ - Respect different viewpoints and experiences.
14
+ - Keep discussions relevant and professional.
15
+
16
+ ## Unacceptable behavior
17
+
18
+ - Harassment, intimidation, discrimination or personal attacks.
19
+ - Sexualized language or unwelcome sexual attention.
20
+ - Insults, demeaning comments or deliberate disruption.
21
+ - Publishing another person's private information without permission.
22
+ - Threats, stalking or sustained unwanted contact.
23
+ - Other conduct that would reasonably make participation unsafe or hostile.
24
+
25
+ ## Scope
26
+
27
+ This Code of Conduct applies in project spaces, including issues, pull requests, discussions, reviews and project-related events or communications. It also applies when someone is representing the project in public.
28
+
29
+ ## Reporting
30
+
31
+ If you experience or witness unacceptable behavior, report it privately to the project maintainers through GitHub. Do not include private information in a public issue. Reports will be handled as confidentially as practical.
32
+
33
+ ## Enforcement
34
+
35
+ Project maintainers are responsible for clarifying and enforcing this Code of Conduct. They may remove or edit comments, reject contributions, temporarily restrict participation, or permanently exclude someone from project spaces when necessary.
36
+
37
+ Maintainers will consider the nature and impact of the behavior, relevant context and any prior pattern. People asked to stop unacceptable behavior are expected to comply immediately.
38
+
39
+ ## Attribution
40
+
41
+ This policy is adapted from the Contributor Covenant, version 2.1:
42
+
43
+ https://www.contributor-covenant.org/version/2/1/code_of_conduct/
@@ -0,0 +1,42 @@
1
+ # Contributing to TwinCheck
2
+
3
+ Thanks for helping improve TwinCheck. The project prioritizes reproducible findings, small understandable changes and honest statements about limitations.
4
+
5
+ ## Before opening an issue
6
+
7
+ - Search existing issues for duplicates.
8
+ - Include TwinCheck version, Python version, operating system, a minimal command, the seed and a sanitized report when possible.
9
+ - Remove credentials, personal data, proprietary inputs and confidential logs before sharing traces.
10
+
11
+ ## Development setup
12
+
13
+ ```bash
14
+ python -m pip install -e .
15
+ python -m unittest discover -s tests -v
16
+ python -m pip install --upgrade build twine
17
+ python -m build --outdir dist
18
+ python scripts/verify_distribution.py --dist-dir dist
19
+ ```
20
+
21
+ The distribution check validates the built wheel and source archive, then installs the wheel into a temporary environment outside the checkout and exercises the CLI entry point. No runtime dependencies are required. Keep the test suite runnable with the Python standard library wherever practical.
22
+
23
+ ## Pull request expectations
24
+
25
+ 1. Describe the behavior changed and why it matters.
26
+ 2. Add a regression test for bug fixes.
27
+ 3. Keep the same seed reproducible and avoid using process-global random state.
28
+ 4. Preserve the principle that reports contain executed evidence, not guessed explanations.
29
+ 5. Update docs and the roadmap when behavior or scope changes.
30
+ 6. Do not introduce network activity or API keys into the core engine.
31
+
32
+ ## Code style
33
+
34
+ Follow standard Python conventions, add types for public interfaces, keep functions focused, and prefer clear error messages over silent fallback. Do not broaden permissions or use `shell=True` to make a command work.
35
+
36
+ ## Release and compatibility policy
37
+
38
+ Before changing CLI flags, report schemas or file formats, read [the release and compatibility policy](docs/release-process.md). TwinCheck is pre-1.0: the Python import API is not stable, and changes to documented interfaces must be reflected in the changelog and relevant schema/version notes. The manual release-candidate workflow builds and verifies artifacts; it intentionally does not publish packages.
39
+
40
+ ## Security-sensitive changes
41
+
42
+ Please read [SECURITY.md](SECURITY.md). Do not run unknown submissions locally. TwinCheck currently executes target commands using the user's privileges and is not a sandbox.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 TwinCheck contributors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,15 @@
1
+ include README.md
2
+ include CHANGELOG.md
3
+ include CONTRIBUTING.md
4
+ include CODE_OF_CONDUCT.md
5
+ include SECURITY.md
6
+ include LICENSE
7
+ include pyproject.toml
8
+
9
+ recursive-include src *.py
10
+ recursive-include tests *.py
11
+ recursive-include scripts *.py
12
+ recursive-include docs *.md *.json
13
+ recursive-include examples *.md *.json
14
+ recursive-include benchmarks *.md *.py *.json
15
+
@@ -0,0 +1,267 @@
1
+ Metadata-Version: 2.4
2
+ Name: twincheck
3
+ Version: 0.12.0a1
4
+ Summary: Reproducible evidence-first differential testing with portable stateful regression workflows.
5
+ Author: TwinCheck contributors
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/ArnabIITP/twincheck
8
+ Project-URL: Repository, https://github.com/ArnabIITP/twincheck
9
+ Project-URL: Issues, https://github.com/ArnabIITP/twincheck/issues
10
+ Project-URL: Documentation, https://github.com/ArnabIITP/TwinCheck/blob/main/docs/quickstart.md
11
+ Project-URL: Changelog, https://github.com/ArnabIITP/TwinCheck/blob/main/CHANGELOG.md
12
+ Project-URL: Security, https://github.com/ArnabIITP/TwinCheck/blob/main/SECURITY.md
13
+ Keywords: differential-testing,black-box-testing,compatibility-testing,api-compatibility,fuzzing,regression-testing,counterexample-minimization,stateful-testing
14
+ Classifier: Development Status :: 3 - Alpha
15
+ Classifier: Environment :: Console
16
+ Classifier: Intended Audience :: Developers
17
+ Classifier: Operating System :: OS Independent
18
+ Classifier: Programming Language :: Python :: 3
19
+ Classifier: Programming Language :: Python :: 3.11
20
+ Classifier: Programming Language :: Python :: 3.12
21
+ Classifier: Programming Language :: Python :: 3.13
22
+ Classifier: Topic :: Software Development :: Testing
23
+ Requires-Python: >=3.11
24
+ Description-Content-Type: text/markdown
25
+ License-File: LICENSE
26
+ Provides-Extra: test
27
+ Requires-Dist: jsonschema<5,>=4.21; extra == "test"
28
+ Dynamic: license-file
29
+
30
+ <div align="center">
31
+
32
+ # TwinCheck
33
+
34
+ **Evidence-first differential testing for software compatibility.**
35
+
36
+ Give TwinCheck a reference implementation and a candidate replacement. It runs both against the same reproducible inputs, compares their observable behavior, and turns differences into evidence you can replay and use in regression testing.
37
+
38
+ [![CI](https://img.shields.io/github/actions/workflow/status/ArnabIITP/TwinCheck/ci.yml?branch=main&label=CI)](https://github.com/ArnabIITP/TwinCheck/actions/workflows/ci.yml)
39
+ [![Python](https://img.shields.io/badge/python-3.11%2B-blue)](https://www.python.org/)
40
+ [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
41
+ [![Project status: alpha](https://img.shields.io/badge/status-alpha-orange)](CHANGELOG.md)
42
+
43
+ </div>
44
+
45
+ > **Alpha software.** TwinCheck is actively developed and its CLI and file formats may evolve before 1.0. It is a testing tool, not a proof of program equivalence, a production monitoring system, or a security sandbox. The first PyPI alpha release is being prepared. Until it is published, use the source installation instructions below. After publication, install the pinned alpha with `python -m pip install twincheck==0.12.0a1`.
46
+
47
+ ## Why TwinCheck?
48
+
49
+ A replacement can pass its unit tests and still behave differently from the system it is meant to replace. Differences may appear in edge cases, error handling, whitespace, status codes, response bodies, session state, or sequences of operations.
50
+
51
+ TwinCheck helps investigate that compatibility gap by comparing two implementations as black boxes. It does not need access to their source code or require them to use the same programming language.
52
+
53
+ Typical use cases include:
54
+
55
+ - Replacing a legacy command-line tool with a new implementation.
56
+ - Porting a parser, formatter, or data-processing utility.
57
+ - Checking whether two local API implementations behave consistently.
58
+ - Testing a migration before switching traffic or removing the old implementation.
59
+ - Preserving a discovered mismatch as a replayable regression case.
60
+
61
+ ## Get started
62
+
63
+ **Requirements:** Python 3.11 or newer.
64
+
65
+ Clone the repository and install the project in editable mode:
66
+
67
+ ```bash
68
+ git clone https://github.com/ArnabIITP/TwinCheck.git
69
+ cd TwinCheck
70
+ python -m pip install -e .
71
+ ```
72
+
73
+ Create a directory for the report files, then run the included command-line example:
74
+
75
+ ```bash
76
+ mkdir -p reports
77
+ twincheck compare \
78
+ --reference "python examples/reference.py" \
79
+ --candidate "python examples/candidate.py" \
80
+ --strategy text \
81
+ --cases 8 \
82
+ --seed 7 \
83
+ --minimize \
84
+ --max-minimization-trials 8 \
85
+ --output reports/demo.json \
86
+ --junit reports/demo-junit.xml
87
+ ```
88
+
89
+ The example candidate deliberately differs from the reference. TwinCheck should report a mismatch and exit with status `1`. That nonzero exit is intentional: a detected difference can fail a CI job. The JSON report contains the executed input and observations; the JUnit file can be consumed by CI test-report tools.
90
+
91
+ For a complete walkthrough, see the **[Quick-start guide](docs/quickstart.md)**.
92
+
93
+ ## What TwinCheck can do today
94
+
95
+ | Area | Current capability |
96
+ | --- | --- |
97
+ | Command-line comparison | Run two commands without a shell, passing the same input through stdin or an argv placeholder |
98
+ | Input generation | Seeded text, integer-boundary, JSON-shaped and documented JSON Schema-subset generation |
99
+ | Known corpora | Replay UTF-8 corpus files in stable path order before generated cases |
100
+ | Behavioral observations | Compare stdout, stderr, exit status, timeout behavior, output truncation and repeated-run instability |
101
+ | Compatibility contracts | Choose exact or structural JSON comparison, ignore selected JSON Pointer paths, and configure exit-code/timeout comparison |
102
+ | Local HTTP comparison | Send matching JSON requests to two loopback services and compare status, selected headers and response bodies |
103
+ | Stateful HTTP workflows | Run ordered multi-request workflows with independent cookies, response-value capture, reset checks and bounded exploration |
104
+ | Failure minimization | Reduce stable mismatches using bounded delta debugging |
105
+ | Replay and regression evidence | Save JSON reports, export portable counterexample packs, and replay findings against explicitly supplied targets |
106
+ | CI reporting | Produce human-readable summaries, JSON evidence and JUnit XML |
107
+ | Python API | Versioned API v1 for command-line comparison, contracts and report helpers |
108
+
109
+ The implementation is deliberately bounded. See [current limitations](docs/roadmap.md) and the [release and compatibility policy](docs/release-process.md) before relying on it in automation.
110
+
111
+ ## How it works
112
+
113
+ 1. **Choose two targets.** Provide the reference and candidate commands, or two local HTTP service URLs.
114
+ 2. **Define the comparison.** Use defaults or a versioned compatibility contract to specify which observations matter.
115
+ 3. **Generate or provide inputs.** Set a seed and optionally provide a corpus or supported JSON Schema.
116
+ 4. **Execute both targets.** TwinCheck runs matching cases and records observable behavior.
117
+ 5. **Inspect differences.** Findings include the input, observations, comparison metadata and provenance.
118
+ 6. **Minimize and replay.** Where supported, shrink stable mismatches and preserve them as regression evidence.
119
+
120
+ ```mermaid
121
+ flowchart TD
122
+ A[Seed, strategy, schema or corpus] --> B[Reproducible inputs]
123
+ B --> C[Reference implementation]
124
+ B --> D[Candidate implementation]
125
+ C --> E[Compare observations]
126
+ D --> E
127
+ E --> F{Difference observed?}
128
+ F -- No --> G[Record completed case]
129
+ F -- Yes --> H[Evidence report]
130
+ H --> I[Optional bounded minimization]
131
+ I --> J[Replayable regression evidence]
132
+ G --> K[Run summary]
133
+ J --> K
134
+ ```
135
+
136
+ A clean run means only that no difference was observed in the cases executed under the selected comparison policy. It does **not** establish equivalence for all possible inputs.
137
+
138
+ ## Examples
139
+
140
+ ### Compare command-line programs
141
+
142
+ ```bash
143
+ twincheck compare \
144
+ --reference "python reference.py" \
145
+ --candidate "python replacement.py" \
146
+ --strategy mixed \
147
+ --cases 500 \
148
+ --seed 42 \
149
+ --repeat 2 \
150
+ --timeout 5 \
151
+ --max-output-bytes 1000000 \
152
+ --minimize \
153
+ --output reports/compatibility.json \
154
+ --junit reports/compatibility.xml
155
+ ```
156
+
157
+ Inputs can be supplied through stdin or a `{input}` argument placeholder. Commands are launched without invoking a shell. See the [CLI guide](docs/quickstart.md#command-line-comparison) for more options.
158
+
159
+ ### Compare local HTTP implementations
160
+
161
+ Start two local example services:
162
+
163
+ ```bash
164
+ python examples/http_service.py --port 8001 --delta 0
165
+ python examples/http_service.py --port 8002 --delta 1
166
+ ```
167
+
168
+ In another terminal:
169
+
170
+ ```bash
171
+ twincheck compare-http \
172
+ --reference-url http://127.0.0.1:8001 \
173
+ --candidate-url http://127.0.0.1:8002 \
174
+ --path /quotes \
175
+ --method POST \
176
+ --schema examples/schemas/quote-request-v1.json \
177
+ --contract examples/contracts/http-api-v1.json \
178
+ --cases 25 \
179
+ --seed 42 \
180
+ --output reports/http-compatibility.json
181
+ ```
182
+
183
+ HTTP targets are restricted to loopback addresses. This adapter is intended for local test services, not remote API scanning.
184
+
185
+ ### Compare stateful API workflows
186
+
187
+ TwinCheck can execute a user-authored sequence such as create a cart, add an item, and read the cart back. It keeps reference and candidate cookies separate, supports scalar response capture for later requests, and can apply a user-defined reset workflow with explicit health assertions.
188
+
189
+ See the [stateful workflow walkthrough](docs/quickstart.md#stateful-http-workflow) and the checked-in [sequence examples](examples/sequences/).
190
+
191
+ ## Python API
192
+
193
+ The supported Python API v1 covers command-line comparisons, compatibility contracts and report helpers:
194
+
195
+ ```python
196
+ import shlex
197
+ import sys
198
+
199
+ from twincheck import compare_commands, write_json_report
200
+
201
+ report = compare_commands(
202
+ shlex.join([sys.executable, "reference.py"]),
203
+ shlex.join([sys.executable, "replacement.py"]),
204
+ cases=100,
205
+ seed=42,
206
+ )
207
+
208
+ write_json_report("reports/compatibility.json", report)
209
+ ```
210
+
211
+ Only documented exports are covered by API v1. Stateful HTTP comparison and exploration are currently CLI features. Read the [Python API reference](docs/python-api.md) and [compatibility policy](docs/release-process.md).
212
+
213
+ ## Benchmarks and evidence
214
+
215
+ The repository includes pinned, synthetic compatibility corpora for cart and booking/reservation workflows. Their fixtures, schemas and expected outcomes are versioned so that results can be reproduced.
216
+
217
+ These are **synthetic validation fixtures**, not independent real-world benchmarks. They do not establish a general detection rate, performance advantage, or production readiness. See the [benchmark protocol](benchmarks/README.md).
218
+
219
+ ## Safety and data handling
220
+
221
+ TwinCheck is **not a sandbox**. It launches configured commands with the permissions of the current user. A target can access files, use the network, spawn processes and consume resources available to that user. Run only commands and services you trust and are authorized to test.
222
+
223
+ Reports, replay artifacts and workflow definitions can contain failing inputs and service observations. Treat them as potentially sensitive. Use synthetic data where possible and review evidence before sharing it. See [SECURITY.md](SECURITY.md) for the project's security boundaries and vulnerability reporting guidance.
224
+
225
+ ## Documentation
226
+
227
+ - [Quick start and CLI walkthrough](docs/quickstart.md)
228
+ - [Python API v1](docs/python-api.md)
229
+ - [JSON Schema generation support](docs/schema-generation.md)
230
+ - [Stateful HTTP sequence format](docs/stateful-http-sequence-v1.schema.json)
231
+ - [Benchmark protocol and limitations](benchmarks/README.md)
232
+ - [Release and compatibility policy](docs/release-process.md)
233
+ - [Architecture and design](docs/architecture.md)
234
+ - [Roadmap and known limitations](docs/roadmap.md)
235
+ - [Security policy](SECURITY.md)
236
+ - [Contributing guide](CONTRIBUTING.md)
237
+ - [Code of Conduct](CODE_OF_CONDUCT.md)
238
+ - [Changelog](CHANGELOG.md)
239
+
240
+ ## Development
241
+
242
+ Install the optional test dependency and run the suite:
243
+
244
+ ```bash
245
+ python -m pip install -e ".[test]"
246
+ python -m unittest discover -s tests -v
247
+ ```
248
+
249
+ Build and verify distributable artifacts:
250
+
251
+ ```bash
252
+ python -m pip install --upgrade build twine
253
+ python -m build --outdir dist
254
+ python scripts/verify_distribution.py --dist-dir dist
255
+ ```
256
+
257
+ See [CONTRIBUTING.md](CONTRIBUTING.md) for development expectations and [docs/release-process.md](docs/release-process.md) for release validation.
258
+
259
+ ## Contributing
260
+
261
+ Bug reports, reproducible examples, documentation improvements and focused pull requests are welcome. Please remove credentials, customer information and proprietary data from reports before sharing them.
262
+
263
+ Start with [CONTRIBUTING.md](CONTRIBUTING.md), review open [issues](https://github.com/ArnabIITP/TwinCheck/issues), and use the issue templates when reporting a problem.
264
+
265
+ ## License
266
+
267
+ TwinCheck is released under the [MIT License](LICENSE).