codereview-loop 0.13.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- codereview_loop-0.13.0/CHANGELOG.md +2303 -0
- codereview_loop-0.13.0/LICENSE +21 -0
- codereview_loop-0.13.0/MANIFEST.in +63 -0
- codereview_loop-0.13.0/PKG-INFO +502 -0
- codereview_loop-0.13.0/README.md +482 -0
- codereview_loop-0.13.0/REVIEW-NOTES.md +98 -0
- codereview_loop-0.13.0/ROADMAP.md +30 -0
- codereview_loop-0.13.0/THREAT-MODEL.md +254 -0
- codereview_loop-0.13.0/bench/BASELINE-2026-11-04.md +282 -0
- codereview_loop-0.13.0/bench/CLAIMS-SHEET.md +204 -0
- codereview_loop-0.13.0/bench/PUBLIC-BENCHMARK.md +457 -0
- codereview_loop-0.13.0/bench/baseline-2026-11-04-results.jsonl +27 -0
- codereview_loop-0.13.0/bench/configs/blind.json +16 -0
- codereview_loop-0.13.0/bench/configs/noisy.json +16 -0
- codereview_loop-0.13.0/bench/configs/oracle.json +16 -0
- codereview_loop-0.13.0/bench/configs/panel-baseline-2026-11-04.json +1 -0
- codereview_loop-0.13.0/bench/configs/panel-gemini-nvidia.json +18 -0
- codereview_loop-0.13.0/bench/corpus/auth-admin-or-true/case.json +24 -0
- codereview_loop-0.13.0/bench/corpus/auth-admin-or-true/diff.patch +8 -0
- codereview_loop-0.13.0/bench/corpus/auth-check-removed/case.json +25 -0
- codereview_loop-0.13.0/bench/corpus/auth-check-removed/diff.patch +9 -0
- codereview_loop-0.13.0/bench/corpus/clean-add-tests/case.json +9 -0
- codereview_loop-0.13.0/bench/corpus/clean-add-tests/diff.patch +14 -0
- codereview_loop-0.13.0/bench/corpus/clean-add-timeout-retry/case.json +9 -0
- codereview_loop-0.13.0/bench/corpus/clean-add-timeout-retry/diff.patch +15 -0
- codereview_loop-0.13.0/bench/corpus/clean-auth-check-added/case.json +9 -0
- codereview_loop-0.13.0/bench/corpus/clean-auth-check-added/diff.patch +9 -0
- codereview_loop-0.13.0/bench/corpus/clean-health-guard-fix/case.json +9 -0
- codereview_loop-0.13.0/bench/corpus/clean-health-guard-fix/diff.patch +29 -0
- codereview_loop-0.13.0/bench/corpus/clean-input-validation/case.json +9 -0
- codereview_loop-0.13.0/bench/corpus/clean-input-validation/diff.patch +11 -0
- codereview_loop-0.13.0/bench/corpus/clean-parameterized-query/case.json +9 -0
- codereview_loop-0.13.0/bench/corpus/clean-parameterized-query/diff.patch +6 -0
- codereview_loop-0.13.0/bench/corpus/clean-refactor-rename/case.json +9 -0
- codereview_loop-0.13.0/bench/corpus/clean-refactor-rename/diff.patch +13 -0
- codereview_loop-0.13.0/bench/corpus/command-injection-shell/case.json +51 -0
- codereview_loop-0.13.0/bench/corpus/command-injection-shell/diff.patch +10 -0
- codereview_loop-0.13.0/bench/corpus/debug-breakpoint/case.json +22 -0
- codereview_loop-0.13.0/bench/corpus/debug-breakpoint/diff.patch +7 -0
- codereview_loop-0.13.0/bench/corpus/delete-no-where/case.json +24 -0
- codereview_loop-0.13.0/bench/corpus/delete-no-where/diff.patch +7 -0
- codereview_loop-0.13.0/bench/corpus/disabled-test/case.json +22 -0
- codereview_loop-0.13.0/bench/corpus/disabled-test/diff.patch +9 -0
- codereview_loop-0.13.0/bench/corpus/drop-table-migration/case.json +23 -0
- codereview_loop-0.13.0/bench/corpus/drop-table-migration/diff.patch +9 -0
- codereview_loop-0.13.0/bench/corpus/error-swallow-bare-except/case.json +24 -0
- codereview_loop-0.13.0/bench/corpus/error-swallow-bare-except/diff.patch +11 -0
- codereview_loop-0.13.0/bench/corpus/eval-user-input/case.json +37 -0
- codereview_loop-0.13.0/bench/corpus/eval-user-input/diff.patch +8 -0
- codereview_loop-0.13.0/bench/corpus/float-money-total/case.json +37 -0
- codereview_loop-0.13.0/bench/corpus/float-money-total/diff.patch +11 -0
- codereview_loop-0.13.0/bench/corpus/http-no-timeout/case.json +23 -0
- codereview_loop-0.13.0/bench/corpus/http-no-timeout/diff.patch +9 -0
- codereview_loop-0.13.0/bench/corpus/manifest.json +222 -0
- codereview_loop-0.13.0/bench/corpus/off-by-one-pagination/case.json +24 -0
- codereview_loop-0.13.0/bench/corpus/off-by-one-pagination/diff.patch +9 -0
- codereview_loop-0.13.0/bench/corpus/path-traversal-upload/case.json +24 -0
- codereview_loop-0.13.0/bench/corpus/path-traversal-upload/diff.patch +13 -0
- codereview_loop-0.13.0/bench/corpus/payment-amount-dollars-cents/case.json +24 -0
- codereview_loop-0.13.0/bench/corpus/payment-amount-dollars-cents/diff.patch +8 -0
- codereview_loop-0.13.0/bench/corpus/payment-quantity-dropped/case.json +23 -0
- codereview_loop-0.13.0/bench/corpus/payment-quantity-dropped/diff.patch +10 -0
- codereview_loop-0.13.0/bench/corpus/payment-refund-partial/case.json +23 -0
- codereview_loop-0.13.0/bench/corpus/payment-refund-partial/diff.patch +7 -0
- codereview_loop-0.13.0/bench/corpus/plaintext-password-store/case.json +37 -0
- codereview_loop-0.13.0/bench/corpus/plaintext-password-store/diff.patch +9 -0
- codereview_loop-0.13.0/bench/corpus/race-check-then-act/case.json +24 -0
- codereview_loop-0.13.0/bench/corpus/race-check-then-act/diff.patch +9 -0
- codereview_loop-0.13.0/bench/corpus/race-counter-increment/case.json +38 -0
- codereview_loop-0.13.0/bench/corpus/race-counter-increment/diff.patch +10 -0
- codereview_loop-0.13.0/bench/corpus/real-buggy-alert-flood/case.json +66 -0
- codereview_loop-0.13.0/bench/corpus/real-buggy-alert-flood/diff.patch +18 -0
- codereview_loop-0.13.0/bench/corpus/secret-aws-key/case.json +23 -0
- codereview_loop-0.13.0/bench/corpus/secret-aws-key/diff.patch +9 -0
- codereview_loop-0.13.0/bench/corpus/secret-hardcoded-api-key/case.json +23 -0
- codereview_loop-0.13.0/bench/corpus/secret-hardcoded-api-key/diff.patch +10 -0
- codereview_loop-0.13.0/bench/corpus/secret-private-key/case.json +23 -0
- codereview_loop-0.13.0/bench/corpus/secret-private-key/diff.patch +11 -0
- codereview_loop-0.13.0/bench/corpus/sensitive-data-logging/case.json +23 -0
- codereview_loop-0.13.0/bench/corpus/sensitive-data-logging/diff.patch +10 -0
- codereview_loop-0.13.0/bench/corpus/sql-injection-concat/case.json +23 -0
- codereview_loop-0.13.0/bench/corpus/sql-injection-concat/diff.patch +8 -0
- codereview_loop-0.13.0/bench/corpus/sql-injection-fstring/case.json +24 -0
- codereview_loop-0.13.0/bench/corpus/sql-injection-fstring/diff.patch +11 -0
- codereview_loop-0.13.0/bench/corpus/ssrf-fetch/case.json +65 -0
- codereview_loop-0.13.0/bench/corpus/ssrf-fetch/diff.patch +10 -0
- codereview_loop-0.13.0/bench/corpus/unbounded-query/case.json +24 -0
- codereview_loop-0.13.0/bench/corpus/unbounded-query/diff.patch +7 -0
- codereview_loop-0.13.0/bench/corpus/xss-innerhtml/case.json +24 -0
- codereview_loop-0.13.0/bench/corpus/xss-innerhtml/diff.patch +9 -0
- codereview_loop-0.13.0/bench/expanded-clean/label-amendments-2026-11-05.md +68 -0
- codereview_loop-0.13.0/bench/expanded-clean/manifest.json +217 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-atomic-state-writer/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-atomic-state-writer/diff.patch +61 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-batch-delete-guarded/case.json +42 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-batch-delete-guarded/diff.patch +49 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-bigdecimal-invoice/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-bigdecimal-invoice/diff.patch +49 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-bounded-response-read/case.json +42 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-bounded-response-read/diff.patch +46 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-circuit-breaker/case.json +71 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-circuit-breaker/diff.patch +87 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-constant-time-compare/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-constant-time-compare/diff.patch +17 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-context-http-timeout/case.json +42 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-context-http-timeout/diff.patch +46 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-csrf-token-check/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-csrf-token-check/diff.patch +37 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-deadletter-worker/case.json +42 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-deadletter-worker/diff.patch +63 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-debounce-search/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-debounce-search/diff.patch +50 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-decimal-money/case.json +42 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-decimal-money/diff.patch +23 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-double-checked-cache/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-double-checked-cache/diff.patch +35 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-errgroup-drain/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-errgroup-drain/diff.patch +29 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-escaped-rendering/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-escaped-rendering/diff.patch +23 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-fetch-timeout-abort/case.json +43 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-fetch-timeout-abort/diff.patch +22 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-graceful-http-shutdown/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-graceful-http-shutdown/diff.patch +73 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-graceful-shutdown/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-graceful-shutdown/diff.patch +64 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-guarded-migration/case.json +42 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-guarded-migration/diff.patch +47 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-health-guard-selfcheck/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-health-guard-selfcheck/diff.patch +86 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-idempotency-keys/case.json +42 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-idempotency-keys/diff.patch +67 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-intl-currency-format/case.json +42 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-intl-currency-format/diff.patch +18 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-jdbc-prepared/case.json +42 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-jdbc-prepared/diff.patch +25 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-locked-counter/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-locked-counter/diff.patch +24 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-mutex-guarded-map/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-mutex-guarded-map/diff.patch +88 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-pagination-bounds/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-pagination-bounds/diff.patch +48 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-password-hash-verify/case.json +71 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-password-hash-verify/diff.patch +58 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-pdo-prepared/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-pdo-prepared/diff.patch +24 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-redacted-logging/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-redacted-logging/diff.patch +49 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-retry-backoff-jitter/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-retry-backoff-jitter/diff.patch +40 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-safe-yaml-config/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-safe-yaml-config/diff.patch +51 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-ssrf-allowlist-fetch/case.json +42 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-ssrf-allowlist-fetch/diff.patch +72 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-strict-json-body/case.json +42 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-strict-json-body/diff.patch +49 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-timeout-http-fetch/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-timeout-http-fetch/diff.patch +46 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-token-bucket-limiter/case.json +11 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-token-bucket-limiter/diff.patch +25 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-tz-aware-scheduling/case.json +71 -0
- codereview_loop-0.13.0/bench/expanded-clean/xc-tz-aware-scheduling/diff.patch +21 -0
- codereview_loop-0.13.0/bench/generation.py +986 -0
- codereview_loop-0.13.0/bench/scripts/__pycache__/run_confirmation_b2.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/bench/scripts/__pycache__/run_confirmation_v2.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/bench/scripts/__pycache__/run_generation_v2.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/bench/scripts/__pycache__/run_panel_baseline.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/bench/scripts/__pycache__/score_confirmation_v2.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/bench/scripts/analyze_failure_features.py +148 -0
- codereview_loop-0.13.0/bench/scripts/blind.json +7 -0
- codereview_loop-0.13.0/bench/scripts/check_archive_suite.py +54 -0
- codereview_loop-0.13.0/bench/scripts/noisy.json +18 -0
- codereview_loop-0.13.0/bench/scripts/oracle.json +521 -0
- codereview_loop-0.13.0/bench/scripts/run_b3_pilot.py +277 -0
- codereview_loop-0.13.0/bench/scripts/run_confirmation_b2.py +227 -0
- codereview_loop-0.13.0/bench/scripts/run_confirmation_v2.py +223 -0
- codereview_loop-0.13.0/bench/scripts/run_generation_v2.py +238 -0
- codereview_loop-0.13.0/bench/scripts/run_panel_baseline.py +174 -0
- codereview_loop-0.13.0/bench/scripts/score_confirmation_v2.py +758 -0
- codereview_loop-0.13.0/bench/scripts/verify_registration.py +409 -0
- codereview_loop-0.13.0/examples/agent-seat-claude.sh +23 -0
- codereview_loop-0.13.0/examples/buggy_alert_flood.patch +18 -0
- codereview_loop-0.13.0/examples/clean_change.patch +29 -0
- codereview_loop-0.13.0/examples/evidence.json +1 -0
- codereview_loop-0.13.0/fixtures-manifest.json +279 -0
- codereview_loop-0.13.0/herdr-plugin-codereview/README.md +103 -0
- codereview_loop-0.13.0/herdr-plugin-codereview/__pycache__/codereview_launch.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/herdr-plugin-codereview/codereview_launch.py +197 -0
- codereview_loop-0.13.0/herdr-plugin-codereview/herdr-plugin.toml +18 -0
- codereview_loop-0.13.0/pilot/DEMO.md +175 -0
- codereview_loop-0.13.0/pilot/FEEDBACK-FORM.md +143 -0
- codereview_loop-0.13.0/pilot/QUICKSTART.md +272 -0
- codereview_loop-0.13.0/pilot/SECURITY-AND-DATA.md +144 -0
- codereview_loop-0.13.0/precedents.json +399 -0
- codereview_loop-0.13.0/probes/README.md +87 -0
- codereview_loop-0.13.0/probes/__pycache__/records_scan.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/probes/closingA_archived_rule.py +52 -0
- codereview_loop-0.13.0/probes/closingA_archived_text_dep.py +73 -0
- codereview_loop-0.13.0/probes/closingA_batch_budget.py +55 -0
- codereview_loop-0.13.0/probes/closingA_function_key.py +72 -0
- codereview_loop-0.13.0/probes/closingA_probe_typeerror.py +130 -0
- codereview_loop-0.13.0/probes/closingA_render_fields.py +69 -0
- codereview_loop-0.13.0/probes/closingA_serving_name.py +72 -0
- codereview_loop-0.13.0/probes/closingA_shadow_cache.py +106 -0
- codereview_loop-0.13.0/probes/closingA_shadow_tuple.py +79 -0
- codereview_loop-0.13.0/probes/closingA_sidecar_sha.py +65 -0
- codereview_loop-0.13.0/probes/closingA_test_coverage.py +78 -0
- codereview_loop-0.13.0/probes/closingA_verification_error_evidence.py +124 -0
- codereview_loop-0.13.0/probes/closingA_verify_chain.py +63 -0
- codereview_loop-0.13.0/probes/closingB_acceptance_m4m6.py +127 -0
- codereview_loop-0.13.0/probes/closingB_bench.py +197 -0
- codereview_loop-0.13.0/probes/closingB_cli_core.py +331 -0
- codereview_loop-0.13.0/probes/closingB_cli_coverage.py +105 -0
- codereview_loop-0.13.0/probes/closingB_disputes_registry.py +110 -0
- codereview_loop-0.13.0/probes/closingB_evidence.py +78 -0
- codereview_loop-0.13.0/probes/closingB_probe_sandbox.py +189 -0
- codereview_loop-0.13.0/probes/closingB_providers.py +139 -0
- codereview_loop-0.13.0/probes/closing_bench_probe.py +121 -0
- codereview_loop-0.13.0/probes/closing_disputes_probe.py +170 -0
- codereview_loop-0.13.0/probes/closing_probe_scripts_probe.py +134 -0
- codereview_loop-0.13.0/probes/closing_records_scan_probe.py +100 -0
- codereview_loop-0.13.0/probes/duplicate_seat_probe.py +136 -0
- codereview_loop-0.13.0/probes/finding_public_probe.py +155 -0
- codereview_loop-0.13.0/probes/fix_verification.py +186 -0
- codereview_loop-0.13.0/probes/gap_attempts_log_probe.py +117 -0
- codereview_loop-0.13.0/probes/gap_bench_force_probe.py +153 -0
- codereview_loop-0.13.0/probes/gap_commonpath_probe.py +122 -0
- codereview_loop-0.13.0/probes/gap_doctor_timeout_probe.py +164 -0
- codereview_loop-0.13.0/probes/gap_matcher_probe.py +68 -0
- codereview_loop-0.13.0/probes/gap_percentile_probe.py +92 -0
- codereview_loop-0.13.0/probes/gap_prompt_slice_probe.py +69 -0
- codereview_loop-0.13.0/probes/gap_providers_probe.py +121 -0
- codereview_loop-0.13.0/probes/gap_quorum_probe.py +98 -0
- codereview_loop-0.13.0/probes/gap_scan_exposure_probe.py +99 -0
- codereview_loop-0.13.0/probes/hardening_parent_fail.py +102 -0
- codereview_loop-0.13.0/probes/lens_mapping_probe.py +132 -0
- codereview_loop-0.13.0/probes/oversize_probe.py +161 -0
- codereview_loop-0.13.0/probes/reachability_probe.py +122 -0
- codereview_loop-0.13.0/probes/records_scan.py +251 -0
- codereview_loop-0.13.0/probes/regeneration_probe.py +216 -0
- codereview_loop-0.13.0/probes/results/closing-bench.txt +12 -0
- codereview_loop-0.13.0/probes/results/closing-disputes.txt +23 -0
- codereview_loop-0.13.0/probes/results/closing-probe-scripts.txt +16 -0
- codereview_loop-0.13.0/probes/results/closing-records-scan.txt +19 -0
- codereview_loop-0.13.0/probes/results/closingA-archived-rule.txt +16 -0
- codereview_loop-0.13.0/probes/results/closingA-archived-text-dep.txt +3 -0
- codereview_loop-0.13.0/probes/results/closingA-batch-budget.txt +3 -0
- codereview_loop-0.13.0/probes/results/closingA-function-key.txt +6 -0
- codereview_loop-0.13.0/probes/results/closingA-probe-typeerror.txt +3 -0
- codereview_loop-0.13.0/probes/results/closingA-render-fields.txt +9 -0
- codereview_loop-0.13.0/probes/results/closingA-serving-name.txt +4 -0
- codereview_loop-0.13.0/probes/results/closingA-shadow-cache.txt +3 -0
- codereview_loop-0.13.0/probes/results/closingA-shadow-tuple.txt +5 -0
- codereview_loop-0.13.0/probes/results/closingA-sidecar-sha.txt +5 -0
- codereview_loop-0.13.0/probes/results/closingA-test-coverage.txt +17 -0
- codereview_loop-0.13.0/probes/results/closingA-verification-error-evidence.txt +9 -0
- codereview_loop-0.13.0/probes/results/closingA-verify-chain.txt +7 -0
- codereview_loop-0.13.0/probes/results/closingB-acceptance-m4m6.txt +23 -0
- codereview_loop-0.13.0/probes/results/closingB-bench.txt +16 -0
- codereview_loop-0.13.0/probes/results/closingB-cli-core.txt +43 -0
- codereview_loop-0.13.0/probes/results/closingB-cli-coverage.txt +21 -0
- codereview_loop-0.13.0/probes/results/closingB-disputes-registry.txt +43 -0
- codereview_loop-0.13.0/probes/results/closingB-evidence.txt +6 -0
- codereview_loop-0.13.0/probes/results/closingB-probe-sandbox.txt +17 -0
- codereview_loop-0.13.0/probes/results/closingB-providers.txt +15 -0
- codereview_loop-0.13.0/probes/results/duplicate-seat-78ddf4d5ea27.txt +24 -0
- codereview_loop-0.13.0/probes/results/finding-public-4ce0212286e0.txt +39 -0
- codereview_loop-0.13.0/probes/results/fix-verification.txt +78 -0
- codereview_loop-0.13.0/probes/results/gap-attempts-log-bca4074a632c.txt +13 -0
- codereview_loop-0.13.0/probes/results/gap-bench-force-474297e6ffdc.txt +8 -0
- codereview_loop-0.13.0/probes/results/gap-commonpath-pair.txt +19 -0
- codereview_loop-0.13.0/probes/results/gap-doctor-timeout-3fc4862824c9.txt +15 -0
- codereview_loop-0.13.0/probes/results/gap-matcher-d93089253022.txt +18 -0
- codereview_loop-0.13.0/probes/results/gap-percentile-09c536d15270.txt +9 -0
- codereview_loop-0.13.0/probes/results/gap-prompt-slice-3f50bf6b3349.txt +16 -0
- codereview_loop-0.13.0/probes/results/gap-providers-cluster.txt +27 -0
- codereview_loop-0.13.0/probes/results/gap-quorum-c5f02551d077.txt +7 -0
- codereview_loop-0.13.0/probes/results/gap-scan-exposure-pair.txt +24 -0
- codereview_loop-0.13.0/probes/results/hardening-parent-fail.txt +18 -0
- codereview_loop-0.13.0/probes/results/lens-mapping-35dbb7d20941.txt +22 -0
- codereview_loop-0.13.0/probes/results/oversize-6219e04d4c00.txt +38 -0
- codereview_loop-0.13.0/probes/results/reachability-f56cd4a0b6c9.txt +30 -0
- codereview_loop-0.13.0/probes/results/records-scan-a3a2d492b51a.txt +115 -0
- codereview_loop-0.13.0/probes/results/regeneration-212878ab6bba.txt +32 -0
- codereview_loop-0.13.0/probes/results/regeneration-7baa0147d8d4.txt +22 -0
- codereview_loop-0.13.0/probes/results/regeneration-7ea4e96603e9.txt +31 -0
- codereview_loop-0.13.0/probes/results/shadow-acceptance-empty-633acb885291.txt +16 -0
- codereview_loop-0.13.0/probes/results/shadow-attach-keys-97feb799979e.txt +13 -0
- codereview_loop-0.13.0/probes/results/shadow-probe-signature-833912a09fa3.txt +14 -0
- codereview_loop-0.13.0/probes/results/shadow-test-runner-4cf81c69d7cf.txt +10 -0
- codereview_loop-0.13.0/probes/results/shadow-trace-gate-4cba208656d3.txt +13 -0
- codereview_loop-0.13.0/probes/results/stub-payload-e1e22d48db4d.txt +32 -0
- codereview_loop-0.13.0/probes/results/unknown-family-3d9645200bc5.txt +33 -0
- codereview_loop-0.13.0/probes/shadow_acceptance_empty_probe.py +156 -0
- codereview_loop-0.13.0/probes/shadow_attach_keys_probe.py +116 -0
- codereview_loop-0.13.0/probes/shadow_probe_signature_probe.py +203 -0
- codereview_loop-0.13.0/probes/shadow_test_runner_probe.py +143 -0
- codereview_loop-0.13.0/probes/shadow_trace_gate_probe.py +110 -0
- codereview_loop-0.13.0/probes/stub_payload_rerun.py +102 -0
- codereview_loop-0.13.0/probes/unknown_family_probe.py +184 -0
- codereview_loop-0.13.0/pyproject.toml +33 -0
- codereview_loop-0.13.0/setup.cfg +4 -0
- codereview_loop-0.13.0/skills/codereview/SKILL.md +131 -0
- codereview_loop-0.13.0/src/codereview/__init__.py +6 -0
- codereview_loop-0.13.0/src/codereview/__main__.py +4 -0
- codereview_loop-0.13.0/src/codereview/acceptance.py +649 -0
- codereview_loop-0.13.0/src/codereview/agentsmd.py +259 -0
- codereview_loop-0.13.0/src/codereview/bench.py +881 -0
- codereview_loop-0.13.0/src/codereview/claims.py +182 -0
- codereview_loop-0.13.0/src/codereview/cli.py +2927 -0
- codereview_loop-0.13.0/src/codereview/core.py +9070 -0
- codereview_loop-0.13.0/src/codereview/crash.py +247 -0
- codereview_loop-0.13.0/src/codereview/discovery.py +289 -0
- codereview_loop-0.13.0/src/codereview/disputes.py +749 -0
- codereview_loop-0.13.0/src/codereview/diversity.py +183 -0
- codereview_loop-0.13.0/src/codereview/evidence.py +1023 -0
- codereview_loop-0.13.0/src/codereview/feedback.py +416 -0
- codereview_loop-0.13.0/src/codereview/github.py +286 -0
- codereview_loop-0.13.0/src/codereview/hardening.py +45 -0
- codereview_loop-0.13.0/src/codereview/herdrhost.py +373 -0
- codereview_loop-0.13.0/src/codereview/hooks.py +261 -0
- codereview_loop-0.13.0/src/codereview/intake.py +208 -0
- codereview_loop-0.13.0/src/codereview/mcp_tools.py +455 -0
- codereview_loop-0.13.0/src/codereview/mcp_transport.py +272 -0
- codereview_loop-0.13.0/src/codereview/notify.py +237 -0
- codereview_loop-0.13.0/src/codereview/pilot.py +549 -0
- codereview_loop-0.13.0/src/codereview/policy.py +1086 -0
- codereview_loop-0.13.0/src/codereview/probe.py +534 -0
- codereview_loop-0.13.0/src/codereview/providers.py +1681 -0
- codereview_loop-0.13.0/src/codereview/record_schema.py +904 -0
- codereview_loop-0.13.0/src/codereview/registry.py +294 -0
- codereview_loop-0.13.0/src/codereview/risk.py +368 -0
- codereview_loop-0.13.0/src/codereview/sarif.py +309 -0
- codereview_loop-0.13.0/src/codereview/seatstate.py +398 -0
- codereview_loop-0.13.0/src/codereview/setup_wizard.py +355 -0
- codereview_loop-0.13.0/src/codereview/support.py +328 -0
- codereview_loop-0.13.0/src/codereview/techlead.py +1041 -0
- codereview_loop-0.13.0/src/codereview/tiering.py +265 -0
- codereview_loop-0.13.0/src/codereview/userconfig.py +307 -0
- codereview_loop-0.13.0/src/codereview_loop.egg-info/PKG-INFO +502 -0
- codereview_loop-0.13.0/src/codereview_loop.egg-info/SOURCES.txt +516 -0
- codereview_loop-0.13.0/src/codereview_loop.egg-info/dependency_links.txt +1 -0
- codereview_loop-0.13.0/src/codereview_loop.egg-info/entry_points.txt +3 -0
- codereview_loop-0.13.0/src/codereview_loop.egg-info/top_level.txt +2 -0
- codereview_loop-0.13.0/src/reviewloop/__init__.py +6 -0
- codereview_loop-0.13.0/src/reviewloop/__main__.py +6 -0
- codereview_loop-0.13.0/tests/__pycache__/test_acceptance_fixes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_agents_md.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_b2_corroborated.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_bench.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_claim_lint.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_claims.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_cli_surface.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_closing_fixes_2.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_core.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_crash_feedback.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_deepseek_connector_switch.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_determinism.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_discovery_wizard.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_diversity.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_error_persistence.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_evidence.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_external_review_v070.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_external_review_v071.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_fixture_secret_hygiene.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_gate_profiles.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_generation.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_generation_driver.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_github.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_harness_fixes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_herdr_plugin.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_herdrhost.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_hooks.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_intake.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_l6_fixes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_local_seats.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_mcp.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_notify.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_p3_01_trust_restriction.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_phase3_security.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_phase4_fixes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_phase4_rerun_fixes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_phase4_residuals.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_phase5_fixes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_phase5_rerun2_fixes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_phase5_rerun_fixes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_phase6_fixes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_phase_l6_validator_hardening.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_pilot_harness.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_policy.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_probe_harness.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_process_tooling.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_profile_mutation_gaps.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_providers.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_ratified_fixes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_record_schema.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_risk.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_role_providers.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_sandbox_c5.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_sarif.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_score_confirmation_v3.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_seat_lenses.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_seatstate.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_shadow_acceptance.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_shadow_p01.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_shadow_p2.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_shadow_tl.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_solution_v073.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_step4_ladder.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_tiering.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_tl_redesign.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_tl_v2_acceptance.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_user_setup.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_v090_fixes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_v091_fixes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_v092_fixes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_v093_fixes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_v093_phase2.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_v094_probes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_v095_fixes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_v096_probes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_v100_integration.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_v111_dispute_probes.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_v111_hardening.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/__pycache__/test_verify_evidence_fix.cpython-312.pyc +0 -0
- codereview_loop-0.13.0/tests/fixtures/herdr/herdr_stub.py +291 -0
- codereview_loop-0.13.0/tests/fixtures/historical-records/README.md +27 -0
- codereview_loop-0.13.0/tests/fixtures/historical-records/review-v096/codereview-20261104-233259.json +5568 -0
- codereview_loop-0.13.0/tests/fixtures/historical-records/review-v15-verify1/codereview-20261104-225424.json +204 -0
- codereview_loop-0.13.0/tests/fixtures/record_schema/blocked.json +295 -0
- codereview_loop-0.13.0/tests/fixtures/record_schema/evidence/93188956c3adf4e6fbd1517046217e27229b7889d4bd6337e9464b664bad8172 +1 -0
- codereview_loop-0.13.0/tests/fixtures/record_schema/inconclusive.json +249 -0
- codereview_loop-0.13.0/tests/fixtures/record_schema/pass.json +212 -0
- codereview_loop-0.13.0/tests/fixtures/seatstate/claude-auth.txt +1 -0
- codereview_loop-0.13.0/tests/fixtures/seatstate/claude-quota.txt +1 -0
- codereview_loop-0.13.0/tests/fixtures/seatstate/codex-loggedout.stderr.txt +34 -0
- codereview_loop-0.13.0/tests/fixtures/seatstate/codex-quota.txt +1 -0
- codereview_loop-0.13.0/tests/fixtures/seatstate/gemini-quota.txt +1 -0
- codereview_loop-0.13.0/tests/fixtures/seatstate/unrelated.txt +3 -0
- codereview_loop-0.13.0/tests/test_acceptance_fixes.py +357 -0
- codereview_loop-0.13.0/tests/test_agents_md.py +353 -0
- codereview_loop-0.13.0/tests/test_b2_corroborated.py +397 -0
- codereview_loop-0.13.0/tests/test_bench.py +262 -0
- codereview_loop-0.13.0/tests/test_claim_lint.py +685 -0
- codereview_loop-0.13.0/tests/test_claims.py +381 -0
- codereview_loop-0.13.0/tests/test_cli_surface.py +84 -0
- codereview_loop-0.13.0/tests/test_closing_fixes_2.py +235 -0
- codereview_loop-0.13.0/tests/test_core.py +714 -0
- codereview_loop-0.13.0/tests/test_crash_feedback.py +412 -0
- codereview_loop-0.13.0/tests/test_deepseek_connector_switch.py +91 -0
- codereview_loop-0.13.0/tests/test_determinism.py +150 -0
- codereview_loop-0.13.0/tests/test_discovery_wizard.py +473 -0
- codereview_loop-0.13.0/tests/test_diversity.py +399 -0
- codereview_loop-0.13.0/tests/test_error_persistence.py +302 -0
- codereview_loop-0.13.0/tests/test_evidence.py +524 -0
- codereview_loop-0.13.0/tests/test_external_review_v070.py +215 -0
- codereview_loop-0.13.0/tests/test_external_review_v071.py +177 -0
- codereview_loop-0.13.0/tests/test_fixture_secret_hygiene.py +81 -0
- codereview_loop-0.13.0/tests/test_gate_profiles.py +484 -0
- codereview_loop-0.13.0/tests/test_generation.py +484 -0
- codereview_loop-0.13.0/tests/test_generation_driver.py +299 -0
- codereview_loop-0.13.0/tests/test_github.py +221 -0
- codereview_loop-0.13.0/tests/test_harness_fixes.py +357 -0
- codereview_loop-0.13.0/tests/test_herdr_plugin.py +492 -0
- codereview_loop-0.13.0/tests/test_herdrhost.py +782 -0
- codereview_loop-0.13.0/tests/test_hooks.py +482 -0
- codereview_loop-0.13.0/tests/test_intake.py +195 -0
- codereview_loop-0.13.0/tests/test_l6_fixes.py +150 -0
- codereview_loop-0.13.0/tests/test_local_seats.py +340 -0
- codereview_loop-0.13.0/tests/test_mcp.py +571 -0
- codereview_loop-0.13.0/tests/test_notify.py +636 -0
- codereview_loop-0.13.0/tests/test_p3_01_trust_restriction.py +423 -0
- codereview_loop-0.13.0/tests/test_phase3_security.py +1077 -0
- codereview_loop-0.13.0/tests/test_phase4_fixes.py +880 -0
- codereview_loop-0.13.0/tests/test_phase4_rerun_fixes.py +272 -0
- codereview_loop-0.13.0/tests/test_phase4_residuals.py +112 -0
- codereview_loop-0.13.0/tests/test_phase5_fixes.py +585 -0
- codereview_loop-0.13.0/tests/test_phase5_rerun2_fixes.py +220 -0
- codereview_loop-0.13.0/tests/test_phase5_rerun_fixes.py +190 -0
- codereview_loop-0.13.0/tests/test_phase6_fixes.py +822 -0
- codereview_loop-0.13.0/tests/test_phase_l6_validator_hardening.py +31 -0
- codereview_loop-0.13.0/tests/test_pilot_harness.py +416 -0
- codereview_loop-0.13.0/tests/test_policy.py +622 -0
- codereview_loop-0.13.0/tests/test_probe_harness.py +535 -0
- codereview_loop-0.13.0/tests/test_process_tooling.py +797 -0
- codereview_loop-0.13.0/tests/test_profile_mutation_gaps.py +437 -0
- codereview_loop-0.13.0/tests/test_providers.py +112 -0
- codereview_loop-0.13.0/tests/test_ratified_fixes.py +354 -0
- codereview_loop-0.13.0/tests/test_record_schema.py +270 -0
- codereview_loop-0.13.0/tests/test_risk.py +524 -0
- codereview_loop-0.13.0/tests/test_role_providers.py +407 -0
- codereview_loop-0.13.0/tests/test_sandbox_c5.py +331 -0
- codereview_loop-0.13.0/tests/test_sarif.py +434 -0
- codereview_loop-0.13.0/tests/test_score_confirmation_v3.py +310 -0
- codereview_loop-0.13.0/tests/test_seat_lenses.py +212 -0
- codereview_loop-0.13.0/tests/test_seatstate.py +428 -0
- codereview_loop-0.13.0/tests/test_shadow_acceptance.py +266 -0
- codereview_loop-0.13.0/tests/test_shadow_p01.py +210 -0
- codereview_loop-0.13.0/tests/test_shadow_p2.py +346 -0
- codereview_loop-0.13.0/tests/test_shadow_tl.py +439 -0
- codereview_loop-0.13.0/tests/test_solution_v073.py +340 -0
- codereview_loop-0.13.0/tests/test_step4_ladder.py +642 -0
- codereview_loop-0.13.0/tests/test_tiering.py +688 -0
- codereview_loop-0.13.0/tests/test_tl_redesign.py +607 -0
- codereview_loop-0.13.0/tests/test_tl_v2_acceptance.py +365 -0
- codereview_loop-0.13.0/tests/test_user_setup.py +323 -0
- codereview_loop-0.13.0/tests/test_v090_fixes.py +773 -0
- codereview_loop-0.13.0/tests/test_v091_fixes.py +454 -0
- codereview_loop-0.13.0/tests/test_v092_fixes.py +202 -0
- codereview_loop-0.13.0/tests/test_v093_fixes.py +460 -0
- codereview_loop-0.13.0/tests/test_v093_phase2.py +681 -0
- codereview_loop-0.13.0/tests/test_v094_probes.py +141 -0
- codereview_loop-0.13.0/tests/test_v095_fixes.py +411 -0
- codereview_loop-0.13.0/tests/test_v096_probes.py +691 -0
- codereview_loop-0.13.0/tests/test_v100_integration.py +519 -0
- codereview_loop-0.13.0/tests/test_v111_dispute_probes.py +247 -0
- codereview_loop-0.13.0/tests/test_v111_hardening.py +204 -0
- codereview_loop-0.13.0/tests/test_verify_evidence_fix.py +228 -0
|
@@ -0,0 +1,2303 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## Unreleased — C1 claim correction + profile-attribution lint (2026-11-07)
|
|
4
|
+
|
|
5
|
+
- **Corrected (defect C1, found by the external tester's
|
|
6
|
+
verification pass):** the README lead paragraph attributed
|
|
7
|
+
the `balanced` profile's registered attempt-1 figures
|
|
8
|
+
(precision 0.875; clean-BLOCKED 22/34 = 0.647, quoted as
|
|
9
|
+
"65%") to the default `strict` gate, and cited a record
|
|
10
|
+
that measured neither. The paragraph now states `strict`'s
|
|
11
|
+
own paired row from the same records — recall 1.00 (7 of 7),
|
|
12
|
+
merged blocking precision 0.583, clean-BLOCKED 32 of 34
|
|
13
|
+
(0.941), each with its Wilson 95% interval — and cites the
|
|
14
|
+
attempt-1 results record,
|
|
15
|
+
`bench/generation-v2/RESULTS-2026-11-05.md`.
|
|
16
|
+
`bench/CLAIMS-SHEET.md` carries
|
|
17
|
+
the paired-`strict` row as a governed claim, with the
|
|
18
|
+
precision derivation stated.
|
|
19
|
+
- The claim lint now checks profile attribution (W0): a
|
|
20
|
+
paragraph that states one profile's recorded figure while
|
|
21
|
+
naming only a different profile — and never the profile of
|
|
22
|
+
record in the paragraph or its section — fails. The
|
|
23
|
+
original C1 sentence is pinned as a mutation test.
|
|
24
|
+
|
|
25
|
+
## Unreleased — L19 collector endpoint live (2026-11-07)
|
|
26
|
+
|
|
27
|
+
- The feedback collector endpoint is set (`feedback.ENDPOINT`
|
|
28
|
+
→ `https://phpatch.com/codereview/report`). The receiving
|
|
29
|
+
service is a small standalone collector on the project host:
|
|
30
|
+
POST-only, the client's own size caps enforced server-side,
|
|
31
|
+
payloads stored under server-generated names and purged
|
|
32
|
+
automatically after 90 days per PRIVACY.md. `report` and the
|
|
33
|
+
opt-in crash ping are live channels now; the manual
|
|
34
|
+
support-bundle route remains available.
|
|
35
|
+
- PRIVACY.md retention wording aligned to the live service
|
|
36
|
+
(90 days, maintainers-only access, automatic purge, deletion
|
|
37
|
+
on request via support@phpatch.com).
|
|
38
|
+
- `CODEREVIEW_RELEASE_CHECK=1` is green with this build.
|
|
39
|
+
|
|
40
|
+
## Unreleased — final build candidate: positioning copy + launch placeholders (2026-11-07)
|
|
41
|
+
|
|
42
|
+
- Positioning copy (owner-approved exact text) is the README
|
|
43
|
+
lead and intro, and the setup wizard's opening line.
|
|
44
|
+
- Launch placeholders filled: security contact
|
|
45
|
+
`support@phpatch.com` with a 5-working-day acknowledgement
|
|
46
|
+
target; privacy / deletion contact `support@phpatch.com`,
|
|
47
|
+
retention 90 days, access limited to the maintainers;
|
|
48
|
+
`[project.urls]` and the report command's tracker URL point
|
|
49
|
+
at the live GitLab project. Provider-family label in the
|
|
50
|
+
crash ping: none (the spec default — the ping carries no
|
|
51
|
+
provider field).
|
|
52
|
+
- Deliberately NOT set: the feedback collector endpoint
|
|
53
|
+
(`feedback.ENDPOINT`). No receiving service exists yet;
|
|
54
|
+
`report` and the crash ping stay inert and the manual
|
|
55
|
+
support-bundle route is the live channel until the owner
|
|
56
|
+
stands one up (it needs its own security review — see the
|
|
57
|
+
L19 request). `CODEREVIEW_RELEASE_CHECK=1` flags exactly
|
|
58
|
+
this one remainder.
|
|
59
|
+
|
|
60
|
+
## Unreleased — crash handling and user feedback (L17–L19)
|
|
61
|
+
|
|
62
|
+
- **L17 — crash handling.** `codereview` now runs under a top-level
|
|
63
|
+
guard (`console_main`). An internal error prints one line, writes
|
|
64
|
+
a redacted `codereview-crash-*.json` note (mode 0600) and exits
|
|
65
|
+
**2**, never 0 or 1. `--debug` / `CODEREVIEW_DEBUG=1` adds the
|
|
66
|
+
redacted traceback. Ctrl-C exits 130; a closed pipe exits 141
|
|
67
|
+
quietly. A crashed `pr --post` run sets the commit status to
|
|
68
|
+
*pending*, never *success*.
|
|
69
|
+
- **L18 — bug reports.** `codereview support-bundle` writes one
|
|
70
|
+
redacted zip (identity, package list, environment variable
|
|
71
|
+
*names*, redacted config and policy, latest record, crash notes,
|
|
72
|
+
MANIFEST). Per-file and total caps; anything left out is listed.
|
|
73
|
+
`codereview report` shows the contents and sends only after
|
|
74
|
+
confirmation, over HTTPS (http only to loopback), without
|
|
75
|
+
following redirects. No endpoint is built in; the owner sets one.
|
|
76
|
+
- **L19 — feedback.** Optional anonymous crash ping, off by
|
|
77
|
+
default (`codereview telemetry enable|disable|status`), fixed
|
|
78
|
+
field list, one per crash signature per day, 3 s timeout, never
|
|
79
|
+
changes the exit code. `CODEREVIEW_TELEMETRY=0` overrides.
|
|
80
|
+
See `PRIVACY.md`, `SECURITY.md`, `CONTRIBUTING.md`.
|
|
81
|
+
- Owner must set before launch: collector endpoint, security and
|
|
82
|
+
privacy contacts, retention text, public tracker URL
|
|
83
|
+
(`CODEREVIEW_RELEASE_CHECK=1` lists what is left).
|
|
84
|
+
|
|
85
|
+
## Unreleased — post-RC fixes from the L6 source review (2026-11-07)
|
|
86
|
+
|
|
87
|
+
The external tester's L6 source review of the 0.13.0 release
|
|
88
|
+
candidate (`69d1c65`, verdict NOT READY) found defects in code
|
|
89
|
+
the earlier phases never exercised. All are fixed here; none
|
|
90
|
+
is a regression from 0.12.1.
|
|
91
|
+
|
|
92
|
+
- **L6 F1/F2 (S1) — the probe tree copy dereferenced
|
|
93
|
+
symlinks.** `run_probe` copied the reviewed tree without
|
|
94
|
+
`symlinks=True`, so a planted link was replaced by its
|
|
95
|
+
target's content inside the sandbox copy (pulling any host
|
|
96
|
+
file the operator can read into the probe tree), a dangling
|
|
97
|
+
link made the copy raise out of `run_probe` — silently
|
|
98
|
+
losing every probe on such a repo — and a link to a special
|
|
99
|
+
file streamed its content into the copy in the parent
|
|
100
|
+
process, unbounded. The copy now preserves links (parity
|
|
101
|
+
with `evidence._copy_tree`), and a copy failure returns an
|
|
102
|
+
`error` probe outcome instead of raising.
|
|
103
|
+
- **L6 F3 (S2) — a non-JSON 2xx reply lost the finished
|
|
104
|
+
review.** `post_review`/`set_status` parsed the response
|
|
105
|
+
body unguarded; an HTML 200 (captive portal, proxy) raised
|
|
106
|
+
`JSONDecodeError` past `review_pr`'s `GitHubError` handler,
|
|
107
|
+
so the CLI died with a traceback and wrote no record. An
|
|
108
|
+
unparseable POST body now raises `GitHubError`, which
|
|
109
|
+
`review_pr` records in the report.
|
|
110
|
+
- **L6 F4 (S3) — `parse_pr_url` accepted malformed URLs.**
|
|
111
|
+
The pattern was not end-anchored (`…/pull/12abc` parsed as
|
|
112
|
+
PR 12) and owner/repo admitted `?`, `#`, `%` into the API
|
|
113
|
+
path. Owner/repo are now `[\w.-]+` and the number must end
|
|
114
|
+
the URL (one trailing slash allowed); suffixed URLs such
|
|
115
|
+
as `/pull/12/files` are rejected, not silently truncated.
|
|
116
|
+
- **L6 F5 (S3) — the auto-stem choice in `write_report` was
|
|
117
|
+
still racy.** The P6-02 exists-check let two processes in
|
|
118
|
+
the same second both pick a free stem, the later
|
|
119
|
+
`os.replace` silently overwriting the earlier record. The
|
|
120
|
+
chosen auto stem is now claimed with an `O_EXCL` claim
|
|
121
|
+
file for the duration of the write; explicit stems (the
|
|
122
|
+
caller's contract) are never claimed or renamed.
|
|
123
|
+
- **L6 F5 rerun (S2, re-rated by the tester) — the claim
|
|
124
|
+
protocol still had a window.** Her rerun reproduced the
|
|
125
|
+
RC losing records (15/15 trials) and the claim build clean
|
|
126
|
+
on her machine (0/15) — but the race is timing-dependent,
|
|
127
|
+
and here the claim build lost records too (up to 4/15 bad
|
|
128
|
+
trials): a rival can claim, write AND release entirely
|
|
129
|
+
inside the window between our exists-check and our claim
|
|
130
|
+
landing, after which our claim succeeds on a stem whose
|
|
131
|
+
record pair now exists and the write overwrites it. The
|
|
132
|
+
pair is now re-validated under the claim; a stem whose
|
|
133
|
+
pair appeared since the check is yielded, never
|
|
134
|
+
overwritten. Pinned by a deterministic interleaving test
|
|
135
|
+
plus her `f5_race.py` (90/90 trials clean after the fix).
|
|
136
|
+
- **L6 rerun S3 — `verify_record` mis-described fresh
|
|
137
|
+
patch-mode records.** A record with no range/head was
|
|
138
|
+
always explained as "predates range/head tracking
|
|
139
|
+
(v0.7.3)"; a record stamped by a build that has the
|
|
140
|
+
tracking (v0.7.3+) but carrying no range is a patch-mode
|
|
141
|
+
run, and now says "no range recorded (patch mode)". Only
|
|
142
|
+
genuinely pre-tracking (or unversioned) records keep the
|
|
143
|
+
age explanation.
|
|
144
|
+
- **Tests/tooling from the same rerun:** a
|
|
145
|
+
mutation-gap pin set for `_apply_gate_profile` /
|
|
146
|
+
`_evidence_class` (her partial mutation pass: 14 of the
|
|
147
|
+
first 30 mutants survived, concentrated in those two
|
|
148
|
+
functions), and `bench/scripts/check_archive_suite.py`,
|
|
149
|
+
which runs the suite from the built delivery archive so a
|
|
150
|
+
count is always measured on the artefact that ships.
|
|
151
|
+
- **Rerun-2 mutation pins (tests only, no behaviour
|
|
152
|
+
change):** her completed mutation pass over `22055d1`
|
|
153
|
+
left 13 of 106 mutants standing. The ones she marked
|
|
154
|
+
worth pinning are now pinned in
|
|
155
|
+
`tests/test_profile_mutation_gaps.py` and verified
|
|
156
|
+
killed under her own harness: `_reviewer_tasks`
|
|
157
|
+
lens-seat pins (mutants 32–35), the `assess_panel`
|
|
158
|
+
detail family list (56, 57) and the
|
|
159
|
+
`escalation_triggers` failed-reviewer naming (93).
|
|
160
|
+
Mutant 54 (`min_families > 0` → `>= 0`) is equivalent —
|
|
161
|
+
the second conjunct is false at 0 — and the `risk.py`
|
|
162
|
+
boundary mutants (68, 81, 84, 86) and the `_gate`
|
|
163
|
+
`allow_major` default (19) remain open as low-value /
|
|
164
|
+
equivalent, as she read them.
|
|
165
|
+
|
|
166
|
+
## 0.13.0 — 2026-11-07
|
|
167
|
+
|
|
168
|
+
First release candidate of the 0.13.0 line. Everything the
|
|
169
|
+
0.13.0.dev0 development window accumulated (the external
|
|
170
|
+
tester's Phases 2–6 follow-ups, the Epic K/L surfaces, the
|
|
171
|
+
B3 prerequisites) is collected in the subsections below,
|
|
172
|
+
followed by the release-candidate prep batch that closes
|
|
173
|
+
the tester's standing must-fix queue (release-gate items
|
|
174
|
+
L8, L10, L11, L13, L15, L16). The default gate profile
|
|
175
|
+
remains `strict`; the B2 registered confirmation ran on
|
|
176
|
+
2026-11-06 and FAILED its primary threshold, and B3 remains
|
|
177
|
+
design-only — no claim in this release rests on either.
|
|
178
|
+
PyPI publication stays on the owner's hold: this release
|
|
179
|
+
exists as a git tag and mirror state, not a package-index
|
|
180
|
+
upload.
|
|
181
|
+
|
|
182
|
+
### Release-candidate prep — L8, L10, L11, L15, L16 (2026-11-07)
|
|
183
|
+
|
|
184
|
+
- **L8 (P3-03 residual) — the bare PEM body shape.**
|
|
185
|
+
`redact_secrets` now redacts a standalone PEM body line
|
|
186
|
+
that carries a key's DER/OpenSSH start (a diff hunk that
|
|
187
|
+
starts mid-key has no header line). Redaction coverage
|
|
188
|
+
26/32 → 27/32 shapes; the five transformed forms remain
|
|
189
|
+
the accepted residual, and README + THREAT-MODEL now
|
|
190
|
+
state the encoded-form boundary plainly.
|
|
191
|
+
- **L10 (P3-12 / NT9) — the notifier's process group dies
|
|
192
|
+
with it.** The summary cap half already landed with
|
|
193
|
+
P3-12 (500 chars, argv and record). The runner now starts
|
|
194
|
+
the command in its own session under a supervised wait
|
|
195
|
+
and kills the whole process group on timeout — and
|
|
196
|
+
sweeps it on normal exit (the P3-06 rule) — instead of
|
|
197
|
+
`subprocess.run`'s direct-child-only kill.
|
|
198
|
+
- **L11 (P3-04 residual) — hardening inside `collect()`.**
|
|
199
|
+
`evidence.collect` applies the process hardening at its
|
|
200
|
+
own entry, so direct library use carries the same
|
|
201
|
+
/proc-environ protection as the CLI path by construction
|
|
202
|
+
(the `_execute` backstop stays). Sibling-process
|
|
203
|
+
exposure remains residual 7, documented and warned.
|
|
204
|
+
- **L15 — the superseded naive precision figure is no
|
|
205
|
+
longer quoted in shipped prose.** It appeared, always
|
|
206
|
+
marked superseded, in `bench/README.md`,
|
|
207
|
+
`bench/PUBLIC-BENCHMARK.md` §5, `bench/CLAIMS-SHEET.md`
|
|
208
|
+
and this CHANGELOG's 0.12.0 entry; all four now refer to
|
|
209
|
+
it by description and point to the corrected counting.
|
|
210
|
+
It survives only in the registered baseline records
|
|
211
|
+
(immutable history) and in the claim lint's detection
|
|
212
|
+
fixture.
|
|
213
|
+
- **L16 — no B2/B3-derived claims; the stale confirmation
|
|
214
|
+
status is corrected.** Shipped docs no longer say the B2
|
|
215
|
+
confirmation "has not run": README, PUBLIC-BENCHMARK
|
|
216
|
+
§3.5 and CLAIMS-SHEET now record the registered FAIL
|
|
217
|
+
(blocking-tier recall 7/24 against ≥ 0.60; clean-BLOCKED
|
|
218
|
+
6/64 passed; default stays `strict`), and B3 is
|
|
219
|
+
described only as design-only and unconfirmed.
|
|
220
|
+
|
|
221
|
+
### L3 smoke fix — the gpt-5 family takes `max_completion_tokens` (2026-11-07)
|
|
222
|
+
|
|
223
|
+
Folded in from the L3 real-system smoke's first pass: the
|
|
224
|
+
`openai` provider preset's default model (gpt-5-mini) could
|
|
225
|
+
not serve through the CLI at all — the provider layer sent
|
|
226
|
+
`max_tokens`, which the gpt-5 family rejects (HTTP 400; it
|
|
227
|
+
requires `max_completion_tokens`), so the seat errored and
|
|
228
|
+
the run degraded; the same seat passed with the documented
|
|
229
|
+
`openai:gpt-4.1` override. Fix: a per-preset
|
|
230
|
+
`completion_tokens` quirk table in the registry (the
|
|
231
|
+
gpt-5 / o-series prefixes on the `openai` preset, empty
|
|
232
|
+
elsewhere), wired through the seat factory beside the
|
|
233
|
+
existing temperature quirk — those models now send the cap
|
|
234
|
+
as `max_completion_tokens`; every other model keeps
|
|
235
|
+
`max_tokens`. Pinned by a payload test per model family
|
|
236
|
+
(no live call). The B2 confirmation's gpt-5.4 seat was
|
|
237
|
+
unaffected: the bench `llm_chain` path builds its own
|
|
238
|
+
payloads.
|
|
239
|
+
|
|
240
|
+
### THREAT-MODEL: accepted record-integrity limit stated; probe paragraph corrected (2026-11-07)
|
|
241
|
+
|
|
242
|
+
Phase 6 rerun 4 verified P6-12b fixed on `35f6043` and
|
|
243
|
+
closed release-gate item L1 on that build. The tester's
|
|
244
|
+
acceptance of the consistent-forgery variants (G2/G3/G4)
|
|
245
|
+
was conditioned on the accepted limit being stated in
|
|
246
|
+
THREAT-MODEL (gate item L12); this entry makes that doc
|
|
247
|
+
change, and corrects one stale paragraph the same
|
|
248
|
+
read-through exposed.
|
|
249
|
+
|
|
250
|
+
- **New residual 13 in `THREAT-MODEL.md`.** The record
|
|
251
|
+
content hash is edit detection, not a signature: it
|
|
252
|
+
detects edits to any record that carries one, and
|
|
253
|
+
verification fails closed on stamping-era or garbled
|
|
254
|
+
declarations — but a record whose declarations are
|
|
255
|
+
rewritten to look pre-stamp is indistinguishable from a
|
|
256
|
+
genuine pre-stamp record, is accepted as legacy, and
|
|
257
|
+
edits to it are not detected. Accepted limit, stated in
|
|
258
|
+
the threat model in the tester's terms.
|
|
259
|
+
- **Probe trust-boundary paragraph corrected.**
|
|
260
|
+
`THREAT-MODEL.md` still said Tech Lead probes set no
|
|
261
|
+
rlimits and kill only the direct child on timeout; the
|
|
262
|
+
C5 hardening (2026-11-06) set rlimits and a process-group
|
|
263
|
+
kill in `probe.py`, so the paragraph now describes the
|
|
264
|
+
sandbox as built. Documentation only — no code changed.
|
|
265
|
+
|
|
266
|
+
### P6-12b: legacy rule applies inside the pre-machinery shape (2026-11-07)
|
|
267
|
+
|
|
268
|
+
The Phase 6 rerun-2 scope boundary exempted any record
|
|
269
|
+
carrying no v1 machinery (`schema_version`, `provenance`,
|
|
270
|
+
`record_identity`) from the P6-12 provenance rule
|
|
271
|
+
categorically. Rerun 3 (the tester's `p6_downgrade3.py`)
|
|
272
|
+
showed the exemption was wider than stated: deleting
|
|
273
|
+
those three sections from a fresh 0.13.0 record — with a
|
|
274
|
+
planted blocker finding added — verified as a "legacy
|
|
275
|
+
record" even while the record still declared
|
|
276
|
+
`version: 0.13.0.dev0` (variant G6), and deleting all
|
|
277
|
+
five declarations verified on no evidence at all (G1).
|
|
278
|
+
|
|
279
|
+
- **The declarations rule now applies to the
|
|
280
|
+
pre-machinery shape too.** `content_hash_required`
|
|
281
|
+
reads the surviving declarations of every hash-less
|
|
282
|
+
record under the same fail-closed rule: legacy needs
|
|
283
|
+
at least one parseable pre-stamp declaration among
|
|
284
|
+
`version`, `provenance.generator_version`,
|
|
285
|
+
`created_at`, and no stamping-era or garbled one.
|
|
286
|
+
G1 and G6 now fail verification; the consistent-
|
|
287
|
+
forgery variants (G2/G3/G4 — forge every declaration
|
|
288
|
+
pre-stamp) still pass, the documented not-a-signature
|
|
289
|
+
limit, unchanged.
|
|
290
|
+
- **Genuine historical records are unaffected.** Every
|
|
291
|
+
writer of the pre-schema era stamped `version` and
|
|
292
|
+
`created_at` (the 0.9.4 / 0.9.6 historical fixtures
|
|
293
|
+
carry both) and they keep the exemption on that
|
|
294
|
+
evidence (pinned).
|
|
295
|
+
- **Pins.** `tests/test_phase6_fixes.py`: the rerun-2
|
|
296
|
+
boundary pin asserted the all-five-deleted shape read
|
|
297
|
+
legacy — that shape is G1, so it is replaced by G1/G6
|
|
298
|
+
failure pins plus a genuine-pre-schema legacy pin
|
|
299
|
+
(+2 net). The hand-built minimal records in
|
|
300
|
+
`tests/test_solution_v073.py`,
|
|
301
|
+
`tests/test_v093_phase2.py` and
|
|
302
|
+
`tests/test_v095_fixes.py` carried no declarations at
|
|
303
|
+
all — a shape no writer ever emitted — and now carry
|
|
304
|
+
the pre-stamp `version`/`created_at` of their era;
|
|
305
|
+
no assertion in them changed.
|
|
306
|
+
- **Release note — the 0.13.0 dev window (noted at the
|
|
307
|
+
tester's rerun-3 request).** Unstamped records written
|
|
308
|
+
by 0.13.0 dev builds between the 0.13.0 version bump
|
|
309
|
+
(2026-11-06 19:16Z) and the content stamp landing
|
|
310
|
+
(2026-11-07 02:07Z) do not read as legacy: their
|
|
311
|
+
version line is stamping-era while their hash was
|
|
312
|
+
never stamped. That closed set is dev artefacts only;
|
|
313
|
+
registered and B2-era records predate the 0.13.0 line
|
|
314
|
+
and are unaffected.
|
|
315
|
+
|
|
316
|
+
### probe sandbox C5 hardening (2026-11-06)
|
|
317
|
+
|
|
318
|
+
The B3 design's hard prerequisite (its §5.1, pass/fail row
|
|
319
|
+
P0): `probe.py`'s sandbox set no rlimits and its timeout
|
|
320
|
+
killed only the direct child, so a forked or backgrounded
|
|
321
|
+
probe process could outlive its budget. Probe execution
|
|
322
|
+
now meets the C5 discipline the evidence sandbox
|
|
323
|
+
established, as a standalone enforcement-only change:
|
|
324
|
+
|
|
325
|
+
- **Process-group kill.** The probe runs in its own
|
|
326
|
+
session; expiry — and normal exit, so a probe that
|
|
327
|
+
backgrounds a child and exits cleanly leaves nothing
|
|
328
|
+
behind — kills the whole process group.
|
|
329
|
+
- **POSIX rlimits** (applied in the child): CPU time =
|
|
330
|
+
wall timeout + 5 s, address space 2 GiB, per-file size
|
|
331
|
+
256 MiB, open files 1024, core dumps off. The process
|
|
332
|
+
count is deliberately not rlimited (a per-user ceiling
|
|
333
|
+
the operator's uid shares); containment is the group
|
|
334
|
+
kill plus the CPU/address-space limits.
|
|
335
|
+
- **Drained, capped output.** Reader threads drain both
|
|
336
|
+
streams under an 8 MiB per-stream cap instead of
|
|
337
|
+
`communicate()`'s unbounded buffering; a capture cut
|
|
338
|
+
short by the cap or by a detached pipe-holder is marked
|
|
339
|
+
in the output, never presented as whole.
|
|
340
|
+
|
|
341
|
+
Outcome vocabulary, the result contract, `probe_decision`,
|
|
342
|
+
`parse_probe_spec` and every gate profile are unchanged —
|
|
343
|
+
a limit-killed probe is an ordinary fail (its exit code is
|
|
344
|
+
the negative signal number), a timed-out probe an ordinary
|
|
345
|
+
timeout. Seven new pins in `tests/test_probe_harness.py`
|
|
346
|
+
cover the limits as observed from inside the probe, the
|
|
347
|
+
forked-child and backgrounded-child kills, the CPU and
|
|
348
|
+
memory ceilings, the file-size ceiling, and the output cap.
|
|
349
|
+
|
|
350
|
+
### B3 dev-pilot scaffolding (2026-11-06)
|
|
351
|
+
|
|
352
|
+
Mechanism-agnostic harness for the B3 dev pilot (design §8).
|
|
353
|
+
Scaffolding and data collection only: no new gate profile, no
|
|
354
|
+
probe-promotion decision, no prediction is computed or
|
|
355
|
+
registered — those wait on the owner's design approval and the
|
|
356
|
+
Q1 protocol ruling.
|
|
357
|
+
|
|
358
|
+
- **Probe channel wired into the bench pipeline (opt-in).**
|
|
359
|
+
`bench.run_bench` accepts `probes=True` (or a config's own
|
|
360
|
+
`"probes": true`) and passes a `probe.make_probe_fn` channel
|
|
361
|
+
into `run_review` — the existing adjudication channel, bounded
|
|
362
|
+
by `MAX_PROBE_ROUNDS`, able only to rebut. Probe outcomes are
|
|
363
|
+
collected onto the result record (`probes_enabled`,
|
|
364
|
+
`probe_outcomes`) as data; wiring the channel changes no
|
|
365
|
+
verdict. Default OFF: the run_review call and the record
|
|
366
|
+
shape are byte-identical to before (pinned by test), and a
|
|
367
|
+
config's identity hash moves only when its probe opt-in is
|
|
368
|
+
enabled.
|
|
369
|
+
- **New module `codereview.pilot`.** The §8 hash split
|
|
370
|
+
(deterministic sha256 partition of case ids into calibration
|
|
371
|
+
/ evaluation slices) with a committed split manifest whose
|
|
372
|
+
`inputs_sha256` binds salt + case set + rule, so a split
|
|
373
|
+
cannot be re-rolled silently (`verify_split_manifest`); the
|
|
374
|
+
§3.1/§3.2 eligibility screen (blocking tier + executable
|
|
375
|
+
basis in the A5 taxonomy + the R4 concrete-trigger test,
|
|
376
|
+
deterministic, basis recorded per finding); and the
|
|
377
|
+
calibration-table builder — per case, findings with
|
|
378
|
+
eligibility and probe outcomes, every current profile's
|
|
379
|
+
verdict replayed through `core.evaluate_gate_policy`, and
|
|
380
|
+
the §8 prediction factors (p_base / p_elig / p_settle) as
|
|
381
|
+
raw input counts per slice. Inputs only: the prediction
|
|
382
|
+
formula and its registration remain the pilot
|
|
383
|
+
registration's step.
|
|
384
|
+
- **Pilot runner `bench/scripts/run_b3_pilot.py`** (preflight /
|
|
385
|
+
run / table): writes the split manifest before anything runs,
|
|
386
|
+
runs the dev sets (v1 public split + expanded stratum) live
|
|
387
|
+
under a current profile with the probe channel optionally
|
|
388
|
+
wired, and emits `calibration-table.json` in the documented
|
|
389
|
+
`codereview.b3-calibration/1` format.
|
|
390
|
+
|
|
391
|
+
### 0.13.0.dev0 (external Phase 6 follow-up fixes)
|
|
392
|
+
|
|
393
|
+
Fixes from the external tester's Phase 6 report (end-to-end
|
|
394
|
+
journeys and non-functional tests, run against ebfa217;
|
|
395
|
+
2026-11-06), each re-verified against the current master —
|
|
396
|
+
P6-01 (the S1) was already fixed by the Phase 5 second-rerun
|
|
397
|
+
batch (atomic record writes) and her crash repro passes
|
|
398
|
+
clean:
|
|
399
|
+
|
|
400
|
+
- **Same-second runs no longer overwrite each other
|
|
401
|
+
(P6-02).** The automatic record stem has one-second
|
|
402
|
+
resolution; two runs in one second on the same `--out`
|
|
403
|
+
silently replaced the first run's record. A taken auto
|
|
404
|
+
stem now gains a counter suffix; explicit stems (the
|
|
405
|
+
post-hooks rewrite contract) are never renamed.
|
|
406
|
+
- **record-verify detects hand-edited records (P6-03).**
|
|
407
|
+
The writer stamps a content hash over the whole record
|
|
408
|
+
(excluding the `record_identity` envelope) and verify
|
|
409
|
+
recomputes it in single and chain mode — a flipped gate,
|
|
410
|
+
a softened finding, forged test results all fail. The
|
|
411
|
+
hash detects edits; it is not a signature (an editor
|
|
412
|
+
who recomputes it passes). Records written before the
|
|
413
|
+
stamp carry no hash and verify as legacy,
|
|
414
|
+
not-applicable — never retro-failed.
|
|
415
|
+
- **Git-quoted paths resolve to their real names
|
|
416
|
+
(P6-04).** Paths git C-quotes (non-ASCII, tab, quote)
|
|
417
|
+
were attributed to the file `unknown`; the patch parser
|
|
418
|
+
now decodes the quoted form everywhere paths are read
|
|
419
|
+
(sections, added lines, line maps, patch stats), so
|
|
420
|
+
per-file coverage, path rules and SARIF locations see
|
|
421
|
+
the real path.
|
|
422
|
+
- **Escalated findings appear in the disputes inventory
|
|
423
|
+
(P6-05).** A finding the Tech Lead escalated to the
|
|
424
|
+
owner was in no open-dispute list: invisible in
|
|
425
|
+
`codereview disputes` and passed over by
|
|
426
|
+
`disputes --check`. `escalated` is now an open
|
|
427
|
+
(human-decision) status there and in the record's own
|
|
428
|
+
`open_disputes` summary; an owner disposition closes it.
|
|
429
|
+
- **Agent-CLI seats carry their CLI's vendor family
|
|
430
|
+
(P6-06).** Every agent seat was family `agent`, so a
|
|
431
|
+
Claude+Codex panel computed as one family and could
|
|
432
|
+
never satisfy two-family release eligibility without a
|
|
433
|
+
hand-written `--seat-meta`. Family now derives from the
|
|
434
|
+
CLI a seat runs (claude → anthropic, codex → openai,
|
|
435
|
+
gemini → google; unknown CLIs keep `agent`), and the
|
|
436
|
+
setup wizard persists the seat metadata it derives
|
|
437
|
+
(`[seat_meta]` in the user config), applied between
|
|
438
|
+
the derived baseline and explicit `--seat-meta`.
|
|
439
|
+
- **The re-ground pass has a call budget (P6-07).**
|
|
440
|
+
Re-grounding fanned out one model call per ungrounded
|
|
441
|
+
finding (3,000 findings → 2,996 calls). The pass stops
|
|
442
|
+
at `MAX_REGROUND_CALLS` (100); findings past the budget
|
|
443
|
+
stay ungrounded with the budget named in their
|
|
444
|
+
re-ground note and in the gate reasons.
|
|
445
|
+
- **`loop`'s surface is documented (P6-08).** The README
|
|
446
|
+
and `loop --help` state what loop takes and what it
|
|
447
|
+
does not (`--seat-meta`/`--policy`/`--role`), and why
|
|
448
|
+
its per-round records — working-tree rounds with no
|
|
449
|
+
committed range/head — are not Form B chain or release
|
|
450
|
+
records. Docs only.
|
|
451
|
+
- **`on_blocked` names a seat state (P6-09).** The hook
|
|
452
|
+
fires when a SEAT ends in a K2 blocked class, not when
|
|
453
|
+
the gate verdict is BLOCKED; the docs now say so, and
|
|
454
|
+
`[hooks]` accepts the additive alias key
|
|
455
|
+
`on_seat_blocked` for the same slot. No rename; the old
|
|
456
|
+
key keeps working.
|
|
457
|
+
- **Record JSON is written with sorted keys (P6-10).**
|
|
458
|
+
- **High-band evidence doctrine aligned (P6-11).** A
|
|
459
|
+
high-band run with no evidence record supplied BLOCKS
|
|
460
|
+
(required evidence *missing*); the docs now state the
|
|
461
|
+
per-item distinction explicitly — a supplied record
|
|
462
|
+
whose required item is *unavailable* is the
|
|
463
|
+
INCONCLUSIVE branch. Code unchanged; a contrast pin
|
|
464
|
+
covers the unavailable case.
|
|
465
|
+
|
|
466
|
+
Rerun addendum (the tester's Phase 6 rerun on the build
|
|
467
|
+
carrying the fixes above confirmed every Phase 6
|
|
468
|
+
disposition, and raised four new items — each re-verified
|
|
469
|
+
here before any change):
|
|
470
|
+
|
|
471
|
+
- **The legacy exemption is bounded by provenance
|
|
472
|
+
(P6-12).** Deleting `content_sha256` from a freshly
|
|
473
|
+
stamped record made record-verify report it as a legacy
|
|
474
|
+
record and pass any edit. A record whose own
|
|
475
|
+
declarations (generator version, creation time) place
|
|
476
|
+
it in the stamping era must now carry the hash; its
|
|
477
|
+
absence fails verification in single and chain mode.
|
|
478
|
+
Records declaring pre-stamp provenance keep the legacy
|
|
479
|
+
reading. The standing limit is unchanged: the hash
|
|
480
|
+
detects edits, it is not a signature.
|
|
481
|
+
- **Hook commands are redacted in the record (P6-13).**
|
|
482
|
+
A `[hooks]` command was stored verbatim in the record's
|
|
483
|
+
hooks section, so a credential embedded in the command
|
|
484
|
+
travelled with the artefact. Stored copies now pass
|
|
485
|
+
through the same secret redaction as captured output;
|
|
486
|
+
the executed command is unchanged.
|
|
487
|
+
- **The sdist ships the guarded docs (P6-14).** The
|
|
488
|
+
claims sheet, the public benchmark doc, the pilot kit
|
|
489
|
+
and the root roadmap/review notes are in the
|
|
490
|
+
distribution, so the unpacked suite runs green and the
|
|
491
|
+
claim lint guards the same doc set everywhere.
|
|
492
|
+
- **Cleanup pins are temp-isolated (P6-15).** The
|
|
493
|
+
sandbox-cleanup tests diffed snapshots of the shared
|
|
494
|
+
system temp dir and could fail on another process's
|
|
495
|
+
sandbox; they now run against a private temp dir.
|
|
496
|
+
|
|
497
|
+
Rerun-2 addendum (the tester's second rerun confirmed
|
|
498
|
+
P6-13, P6-14 and P6-15 fixed, and showed P6-12 only
|
|
499
|
+
partly closed):
|
|
500
|
+
|
|
501
|
+
- **The legacy exemption now fails closed (P6-12,
|
|
502
|
+
rerun 2).** The provenance bound trusted any one
|
|
503
|
+
declaration and read missing or unparseable fields as
|
|
504
|
+
pre-stamp, so stripping the hash plus deleting or
|
|
505
|
+
garbling two fields passed an edited record as legacy
|
|
506
|
+
(seven variants). All three declarations (version,
|
|
507
|
+
`provenance.generator_version`, `created_at`) are now
|
|
508
|
+
read and classified; legacy requires positive,
|
|
509
|
+
consistent pre-stamp evidence, and a stamping-era,
|
|
510
|
+
garbled or absent provenance fails verification in
|
|
511
|
+
single and chain mode. Honest pre-stamp records keep
|
|
512
|
+
the exemption. Records carrying no trace of the v1
|
|
513
|
+
machinery at all (the pre-schema shape) stay
|
|
514
|
+
not-applicable, as for every machinery-era check;
|
|
515
|
+
reaching that shape from a current record means
|
|
516
|
+
deleting its entire format layer, the documented
|
|
517
|
+
wholesale-rewrite residual. One documented cost: unstamped records
|
|
518
|
+
from 0.13.0 dev builds in the hours before the stamp
|
|
519
|
+
landed no longer read as legacy. The standing limit is
|
|
520
|
+
unchanged: the hash detects edits, it is not a
|
|
521
|
+
signature.
|
|
522
|
+
- **Invalid escape fixed (rerun-2 nit).** The redaction
|
|
523
|
+
stop-char literal spelled a backslash as an invalid
|
|
524
|
+
escape sequence (a SyntaxWarning on Python 3.12+, an
|
|
525
|
+
error under `-W error`); the set is unchanged, the
|
|
526
|
+
spelling is now correct, and a compile pin holds it.
|
|
527
|
+
|
|
528
|
+
### 0.13.0.dev0 (external Phase 5 second-rerun fixes)
|
|
529
|
+
|
|
530
|
+
Fixes from the external tester's Phase 5 second rerun on
|
|
531
|
+
the fixed build 784ff95 (2026-11-06), each independently
|
|
532
|
+
verified against master. The rerun verified all four
|
|
533
|
+
previous residuals fixed; what remained was in the claim
|
|
534
|
+
lint, plus the write-atomicity remainder of P6-01 (the
|
|
535
|
+
same write path P5-07b hardened, fixed early ahead of the
|
|
536
|
+
Phase 6 follow-up):
|
|
537
|
+
|
|
538
|
+
- **Record writes are atomic (P6-01).** `write_report`
|
|
539
|
+
wrote the final path in place: a write failing midway
|
|
540
|
+
(file-size limit, disk full, a kill) left a truncated,
|
|
541
|
+
invalid JSON record at the `--out` path — the P5-07b
|
|
542
|
+
fallback net preserved the review, but a tool scanning
|
|
543
|
+
`--out` still picked up the broken file. Both the JSON
|
|
544
|
+
record and the Markdown report now land via a temp file
|
|
545
|
+
in the same directory, fsync, and `os.replace`; on
|
|
546
|
+
failure the temp is removed and the final path holds the
|
|
547
|
+
previous file or nothing. The hooks' rewrite of the
|
|
548
|
+
record (the record is rewritten at the same stem once
|
|
549
|
+
the post-record hooks have fired) goes through the same
|
|
550
|
+
writer, so a midway rewrite failure leaves the first,
|
|
551
|
+
complete record intact.
|
|
552
|
+
- **Claim lint: six more evasions closed (P5-08c).**
|
|
553
|
+
Paragraphs are matched NFKC-normalised, so full-width
|
|
554
|
+
digits and letters can no longer disguise a figure;
|
|
555
|
+
the pattern table gains the count-free all-of-set
|
|
556
|
+
form, the spelled-out percent form, and the remaining
|
|
557
|
+
perfection phrasings the second rerun tried; and the
|
|
558
|
+
fraction rule accepts the spelled N-of-N form under
|
|
559
|
+
the same local-context discipline. The guarded
|
|
560
|
+
shipped docs pass unedited, and the tester's three
|
|
561
|
+
lint scripts are now fully caught — the newest set
|
|
562
|
+
went from four of ten caught to ten of ten.
|
|
563
|
+
- **The claim lint states its own limit (P5-08d).** The
|
|
564
|
+
lint's docstring now says plainly that it is a phrase
|
|
565
|
+
list — a tripwire over known phrasings, not a proof a
|
|
566
|
+
document makes no unqualified claim — and that any new
|
|
567
|
+
wording passes until it is added.
|
|
568
|
+
|
|
569
|
+
### 0.13.0.dev0 (external Phase 5 rerun fixes)
|
|
570
|
+
|
|
571
|
+
Fixes from the external tester's Phase 5 rerun on the fixed
|
|
572
|
+
build ebfa217 (2026-11-06), each independently verified
|
|
573
|
+
against master:
|
|
574
|
+
|
|
575
|
+
- **The `--out` check proves writability, and a late write
|
|
576
|
+
failure keeps the review (P5-07b).** The Phase 5 up-front
|
|
577
|
+
check inferred writability from `os.access`, which passes
|
|
578
|
+
for a root user on `/proc` and `/sys`: the full review ran,
|
|
579
|
+
then the report write crashed and discarded the result.
|
|
580
|
+
Three prongs: (1) the check now proves the directory by
|
|
581
|
+
creating the directory chain and a temp file for real,
|
|
582
|
+
then removing them — a create fails for root exactly as
|
|
583
|
+
for anyone else — so an unusable `--out` still refuses
|
|
584
|
+
before any provider call; (2) independently, the report
|
|
585
|
+
write is wrapped: an `OSError` after the review completes
|
|
586
|
+
prints the verdict, writes the record to a fallback
|
|
587
|
+
location (current directory, then temp directory, then
|
|
588
|
+
stdout itself) and exits 2 instead of tracebacking;
|
|
589
|
+
(3) the tests simulate the unwritable case by mocking the
|
|
590
|
+
create, so they pass identically as root and non-root.
|
|
591
|
+
- **The PR reasons line carries no live Markdown (P5-04b).**
|
|
592
|
+
Gate reasons are untrusted text (they can carry
|
|
593
|
+
repo-policy-derived content), and the Phase 5
|
|
594
|
+
neutralisation left bare URLs autolinking, `#N` and
|
|
595
|
+
`owner/repo#N` forming cross-references, and
|
|
596
|
+
reference-style links live. The rendering boundary now
|
|
597
|
+
also breaks the scheme separator in bare URLs, splits
|
|
598
|
+
the reference-link join, defuses line-start reference
|
|
599
|
+
definitions, and breaks `#` before a digit.
|
|
600
|
+
- **Claim lint: wider scope, more phrasings (P5-08b).**
|
|
601
|
+
The lint's doc set is now every shipped *.md in the tree
|
|
602
|
+
(docs/, skills/, plugin and probe READMEs included),
|
|
603
|
+
with records excluded by name and by tree under the same
|
|
604
|
+
immutability rule as before; the pattern table gains the
|
|
605
|
+
reversed-decimal, spelled-out, all-of-set and
|
|
606
|
+
no-miss phrasing families the rerun evaded with, under
|
|
607
|
+
the same same-paragraph qualification and local-window
|
|
608
|
+
discipline.
|
|
609
|
+
- **SARIF `..` segments resolve (P5-01b).** The Phase 5
|
|
610
|
+
normalisation dropped `..` segments, which could point a
|
|
611
|
+
finding at a different file. Paths now resolve with
|
|
612
|
+
`posixpath.normpath`; a path that still escapes the root
|
|
613
|
+
after normalisation emits no location at all rather
|
|
614
|
+
than a misleading one.
|
|
615
|
+
- **P5-14 recheck: not a defect.** The rerun's capture
|
|
616
|
+
showed the herdr launcher's file-path message clipped;
|
|
617
|
+
the literal in the launcher source is intact ("workspace
|
|
618
|
+
path is not a directory") — the tester's harness cut the
|
|
619
|
+
line. Pinned in the rerun regression tests so the
|
|
620
|
+
wording cannot silently regress.
|
|
621
|
+
|
|
622
|
+
### 0.13.0.dev0 (external Phase 4 rerun fixes)
|
|
623
|
+
|
|
624
|
+
Fixes from the external tester's Phase 4 rerun on the fixed
|
|
625
|
+
build (2026-11-06), each independently verified against
|
|
626
|
+
master. The fix implementations for the four code items were
|
|
627
|
+
adopted from the tester's own solution patch (p4-residuals),
|
|
628
|
+
with completions noted:
|
|
629
|
+
|
|
630
|
+
- **Torn JSONL tails are removed before appending
|
|
631
|
+
(P4-08b).** The P4-08 read tolerance left the torn bytes in
|
|
632
|
+
place: the first resume appended after them, and the
|
|
633
|
+
second resume crashed on the fragment turned corrupt
|
|
634
|
+
mid-file line. Before the generation/attempt/journal
|
|
635
|
+
append handles open, a torn final line is now dropped
|
|
636
|
+
(temp file + fsync + atomic replace) and an unterminated
|
|
637
|
+
parseable final line gains its newline; when dropping the
|
|
638
|
+
tail would expose another invalid line, the file is left
|
|
639
|
+
untouched — real damage still raises on read.
|
|
640
|
+
- **Provider and seat error text is redacted before it can
|
|
641
|
+
reach a record (P4-01b).** The P4-01 error-body excerpt
|
|
642
|
+
entered the ProviderError message unredacted, and that
|
|
643
|
+
message is persisted verbatim into records and reports —
|
|
644
|
+
a 401 body quoting the caller's own key back leaked it.
|
|
645
|
+
The excerpt now passes through `redact_secrets` before
|
|
646
|
+
truncation (class-deciding text such as
|
|
647
|
+
`insufficient_quota` survives); if redaction itself fails
|
|
648
|
+
the excerpt is dropped and the status kept. Audit
|
|
649
|
+
extension: an agent seat's stderr tail, quoted into its
|
|
650
|
+
error on a non-zero exit, is redacted at the same
|
|
651
|
+
boundary.
|
|
652
|
+
- **JSON error envelopes are scanned by value (P4-06b).**
|
|
653
|
+
A line that is a JSON error envelope (`is_error: true`,
|
|
654
|
+
or an `error` key with no verdict fields) is matched
|
|
655
|
+
against its decoded string values, so a CLI's own failure
|
|
656
|
+
text is no longer rejected for sitting inside JSON quotes.
|
|
657
|
+
Keys are never scanned, and a verdict object — whose free
|
|
658
|
+
text quotes the diff under review — is never
|
|
659
|
+
value-scanned.
|
|
660
|
+
- **Quotation lead-ins do not classify (P4-06c).** A line
|
|
661
|
+
whose first non-space character is `+`, `>` or `|` (diff /
|
|
662
|
+
markdown quoting), and a lead-in containing a code call
|
|
663
|
+
(`print(...)`), are content under review, not the CLI
|
|
664
|
+
speaking. `-` is deliberately allowed (an ordinary vendor
|
|
665
|
+
bullet). Accepted residual, documented in seatstate.py:
|
|
666
|
+
the bare unquoted third-party state report ("MCP server
|
|
667
|
+
docs not logged in; continuing") still matches — the
|
|
668
|
+
`; continuing` lookahead that would catch it encodes one
|
|
669
|
+
vendor's phrasing and was rejected on that basis.
|
|
670
|
+
- **Suite portability.** `test_score_confirmation_v3` now
|
|
671
|
+
skips cleanly when the swarm environment's `llm_chain`
|
|
672
|
+
module is unavailable (it drives bench scripts that import
|
|
673
|
+
it from outside the repo), matching the guard its two
|
|
674
|
+
sibling modules already carry — a clean checkout reports
|
|
675
|
+
the suite green instead of one import error.
|
|
676
|
+
|
|
677
|
+
### 0.13.0.dev0 (external Phase 5 test fixes)
|
|
678
|
+
|
|
679
|
+
Fixes from the external tester's Phase 5 report (2026-11-06 —
|
|
680
|
+
interfaces and protocol conformance), each independently
|
|
681
|
+
verified against master before landing. Two items needed no
|
|
682
|
+
fix: the missing/wrong-`jsonrpc` half of P5-05 was already
|
|
683
|
+
closed by P3-10, and P5-09's help exit codes by D-004/D-005
|
|
684
|
+
(now pinned for every subcommand):
|
|
685
|
+
|
|
686
|
+
- **SARIF conformance (P5-01).** `artifactLocation.uri` is
|
|
687
|
+
now a percent-encoded relative URI reference: drive
|
|
688
|
+
letters, leading slashes and `.`/`..` segments are dropped
|
|
689
|
+
rather than resolved, and each remaining segment is
|
|
690
|
+
percent-encoded. `startLine` is clamped to a positive
|
|
691
|
+
int32, and control characters are stripped from
|
|
692
|
+
`message.text`. Fingerprints and same-input determinism
|
|
693
|
+
are unchanged.
|
|
694
|
+
- **PR posting boundary (P5-02, P5-03, P5-04, P5-11).**
|
|
695
|
+
`pr --post` without `GITHUB_TOKEN` fails fast with exit 2
|
|
696
|
+
before any provider call. A comment or status POST failure
|
|
697
|
+
no longer discards the completed review: the failure and
|
|
698
|
+
what was already posted are recorded on the report
|
|
699
|
+
(`posting`), the local report is written, and the exit
|
|
700
|
+
code still reflects the posting failure. Rendered PR
|
|
701
|
+
comments neutralise untrusted model/diff-derived text
|
|
702
|
+
(@mentions, markdown images and links, raw HTML render
|
|
703
|
+
inert; the comment's own structure is untouched). The
|
|
704
|
+
inconclusive gate keeps its documented `pending` commit
|
|
705
|
+
status, and the status description now says the gate was
|
|
706
|
+
inconclusive, explicitly.
|
|
707
|
+
- **MCP envelope validation (P5-05).** A request id that is
|
|
708
|
+
a bool, float, null, object or array is an invalid request
|
|
709
|
+
(-32600, answered with a null id); string and integer ids
|
|
710
|
+
behave as before, and an absent id keeps notification
|
|
711
|
+
semantics.
|
|
712
|
+
- **Wizard interrupts (P5-06, P5-13).** Ctrl-C in
|
|
713
|
+
`codereview setup` prints "cancelled — nothing saved." and
|
|
714
|
+
exits 130 — no traceback, no config written. The EOF path
|
|
715
|
+
prints its abort message once, not twice.
|
|
716
|
+
- **Unwritable `--out` refused up front (P5-07).** Every
|
|
717
|
+
command that accepts `--out` (main gate, `pr`, `loop`,
|
|
718
|
+
`workflow`, `intake`, `bench`) validates the target at
|
|
719
|
+
argument time and exits 2 before any provider call, where
|
|
720
|
+
it previously ran the full review and then lost the
|
|
721
|
+
result to a raw traceback.
|
|
722
|
+
- **Claim lint (P5-08).** The claim lint is now
|
|
723
|
+
paragraph-level over all shipped reader docs (registered
|
|
724
|
+
run records excluded by name as immutable), with a pattern
|
|
725
|
+
table covering decimal, percentage and phrasing variants
|
|
726
|
+
of the headline figures. All 11 of the tester's
|
|
727
|
+
unqualified mutation variants now fail it; existing docs
|
|
728
|
+
were qualified in place, never the lint weakened.
|
|
729
|
+
- **Docs and wording (P5-10, P5-12, P5-14).** Hook
|
|
730
|
+
positional arguments are documented where hooks are
|
|
731
|
+
documented (a bare script path receives none; a shell
|
|
732
|
+
string forwards `"$1" "$2"`), the MCP server's
|
|
733
|
+
single-threaded cancellation limit is documented, and the
|
|
734
|
+
herdr plugin launcher says "is not a directory" for a
|
|
735
|
+
path that exists but is a file.
|
|
736
|
+
|
|
737
|
+
### 0.13.0.dev0 (external Phase 4 test fixes)
|
|
738
|
+
|
|
739
|
+
Fixes from the external tester's Phase 4 report (2026-11-06 —
|
|
740
|
+
provider failure totality), each independently verified
|
|
741
|
+
against master before landing:
|
|
742
|
+
|
|
743
|
+
- **Provider transport — quota, deadlines, garbage (P4-01,
|
|
744
|
+
P4-02, P4-03, P4-04, P4-05).** API error responses now read
|
|
745
|
+
their bodies (bounded excerpt), so a 429 whose body names a
|
|
746
|
+
quota classifies `quota` on the first attempt instead of
|
|
747
|
+
burning three rate-limit retries; `Retry-After` (seconds or
|
|
748
|
+
HTTP-date) floors the retry delay, capped at 120s. Every
|
|
749
|
+
API call carries an overall deadline driven by the
|
|
750
|
+
operator's `--timeout` (a trickling response can no longer
|
|
751
|
+
pin a seat forever) and a 16 MiB streamed response cap.
|
|
752
|
+
Transport-level garbage — malformed JSON, an HTML 200,
|
|
753
|
+
non-UTF-8 bytes, a truncated body — surfaces as a transient
|
|
754
|
+
`ProviderError` and is retried, never again mistaken for a
|
|
755
|
+
model-output parse failure. Empty-content responses carry
|
|
756
|
+
`kind="empty"` end to end, so the review retry layer and
|
|
757
|
+
the benchmark's generation seats treat them as the content
|
|
758
|
+
class they are (re-draw) instead of `unknown`/exhausted.
|
|
759
|
+
- **Seat-state anchoring (P4-06).** Vendor failure patterns
|
|
760
|
+
now count only on lines where the CLI speaks in its own
|
|
761
|
+
unquoted voice: a benign warning quoting a component's
|
|
762
|
+
state (`warning: MCP server 'docs' not logged in;
|
|
763
|
+
continuing`) no longer early-terminates a healthy seat as
|
|
764
|
+
`blocked:auth`, and a crashed run whose stdout quotes
|
|
765
|
+
failure text from a reviewed diff no longer classifies
|
|
766
|
+
`blocked:auth`. The verbatim vendor lines — including
|
|
767
|
+
timestamped logger-prefixed captures — still classify, and
|
|
768
|
+
Claude's failure-as-result stdout channel is preserved.
|
|
769
|
+
- **Herdr watch backoff (P4-07).** Consecutive pane-get
|
|
770
|
+
failures in the hosted watch back off exponentially
|
|
771
|
+
(0.05s → 2s cap, reset on success) instead of spawning
|
|
772
|
+
herdr ~13 times a second; a bounded streak of 20 with
|
|
773
|
+
reconciliation still proving the session healthy ends the
|
|
774
|
+
seat with the recorded herdr-unreachable outcome.
|
|
775
|
+
- **Benchmark durability (P4-08, P4-09).** A torn final JSONL
|
|
776
|
+
line in the generation driver's records is skipped on
|
|
777
|
+
resume (corruption anywhere else still raises). Seat
|
|
778
|
+
metering is journaled at call time
|
|
779
|
+
(`events-attempt-N.jsonl`) and merged into the record fold
|
|
780
|
+
on resume, so a kill between the cache write and the fold
|
|
781
|
+
no longer understates a case's attempts and tokens.
|
|
782
|
+
- **Future-dated waivers (P4-10).** A waiver dated more than
|
|
783
|
+
one day in the future is now a `waivers.max_age_days`
|
|
784
|
+
violation — it previously passed the age check trivially
|
|
785
|
+
(negative age) and waived findings forever. One day of
|
|
786
|
+
tolerance absorbs timezone skew from date-only parsing.
|
|
787
|
+
- **Single-family panel label (P4-11).** A PASS from a
|
|
788
|
+
multi-seat panel whose serving seats all share one known
|
|
789
|
+
model family now carries "single-family panel — no
|
|
790
|
+
independent review" in its gate reasons, mirroring the
|
|
791
|
+
solo-panel label. No verdict change.
|
|
792
|
+
|
|
793
|
+
### 0.13.0.dev0 (external Phase 3 test fixes)
|
|
794
|
+
|
|
795
|
+
Fixes from the external tester's Phase 3 (security) report
|
|
796
|
+
(2026-11-06), each independently verified against master
|
|
797
|
+
before landing:
|
|
798
|
+
|
|
799
|
+
- **P3-01 — policy provenance is observable (observability
|
|
800
|
+
only).** The run record's policy section carries an
|
|
801
|
+
`origin` (`discovered-in-repo` / `explicit` / `none`),
|
|
802
|
+
the CLI warns on stderr when a repo-discovered policy is
|
|
803
|
+
applied, and a change that itself adds or modifies
|
|
804
|
+
`.codereview.toml` / `AGENTS.md` is flagged in the
|
|
805
|
+
record's `policy_self_modification` list (from patch
|
|
806
|
+
sections, plus a git-status check for the untracked-
|
|
807
|
+
policy case in working-tree mode). Which policy keys
|
|
808
|
+
apply, and whether hooks run, are UNCHANGED — restricting
|
|
809
|
+
a discovered policy's authority awaits an owner decision.
|
|
810
|
+
- **P3-01 — the trust restriction has landed (owner ruling
|
|
811
|
+
2026-11-06, option 1: restrict).** A policy DISCOVERED in
|
|
812
|
+
the reviewed repository now keeps only its strengthening
|
|
813
|
+
keys. Its `[hooks]` (commands and timeout), a
|
|
814
|
+
weaker-than-`strict` `[gate]` profile, `[[paths]]
|
|
815
|
+
enforce = "advisory"`, the `[waivers]` leniency values,
|
|
816
|
+
`[diversity] enforce = "degrade"` /
|
|
817
|
+
`exclude_builder_family`, and `[risk]` tiering with its
|
|
818
|
+
seat selection are dropped back to defaults at load —
|
|
819
|
+
before the hook runner is built or the gate reads the
|
|
820
|
+
policy — enumerated in one audit point
|
|
821
|
+
(`policy.DISCOVERED_POLICY_RESTRICTED_KEYS`, enforced by
|
|
822
|
+
`policy.apply_discovered_restrictions`). Dropped keys are
|
|
823
|
+
named in the record's `policy.ignored_keys`, counted in
|
|
824
|
+
the CLI's discovered-policy warning (which now names the
|
|
825
|
+
remedy), and returned to MCP clients as
|
|
826
|
+
`policy_ignored_keys`. New `--trust-repo-policy` flag
|
|
827
|
+
(CLI, and `run_gate`'s `trust_repo_policy` argument)
|
|
828
|
+
honours a discovered policy in full; an operator-named
|
|
829
|
+
`--policy` is honoured in full exactly as before.
|
|
830
|
+
THREAT-MODEL gains the "policy may come from the reviewed
|
|
831
|
+
tree" trust-boundary entry.
|
|
832
|
+
- **P3-02 — the deterministic scanner is no longer cubic.**
|
|
833
|
+
Every unbounded gap in the `sql-string-build` rule is
|
|
834
|
+
bounded to 200 chars and a single added line is scanned
|
|
835
|
+
to at most 16,384 chars; the tester's 18 KB pathological
|
|
836
|
+
line went from ~31 s to ~0.1 s, with legitimate SQL
|
|
837
|
+
string-build findings unchanged.
|
|
838
|
+
- **P3-03 — redaction covers the token formats it meets.**
|
|
839
|
+
PEM blocks (whole block), AWS AKIA/ASIA ids, GitHub
|
|
840
|
+
ghp_/github_pat_, OpenAI/Anthropic sk- shapes, Stripe
|
|
841
|
+
live/test keys, Slack xox tokens, Google API keys, JWTs,
|
|
842
|
+
URL credentials, `Authorization: Basic`, and keyword
|
|
843
|
+
assignments in unquoted, quoted-key, JSON and
|
|
844
|
+
string-literal-value forms (value floor lowered to 4).
|
|
845
|
+
The scanner flags the same provider prefixes. Redaction
|
|
846
|
+
remains pattern-based best-effort (the README now says
|
|
847
|
+
so); generically encoded forms (base64 / hex /
|
|
848
|
+
URL-encoded / line-wrapped copies of a key) are a
|
|
849
|
+
documented residual.
|
|
850
|
+
- **P3-04 — the gate process makes itself non-dumpable on
|
|
851
|
+
Linux** (prctl PR_SET_DUMPABLE), at CLI start, at MCP
|
|
852
|
+
server start, and lazily in the evidence executor and
|
|
853
|
+
the discovery probe — an env-only sandbox child can no
|
|
854
|
+
longer read the gate's environment out of
|
|
855
|
+
/proc/<pid>/environ.
|
|
856
|
+
- **P3-05 — a deeply nested MCP frame no longer kills the
|
|
857
|
+
server**: RecursionError at parse time is answered
|
|
858
|
+
-32700, recorded in the errors ledger, and serving
|
|
859
|
+
continues.
|
|
860
|
+
- **P3-06 — sandbox runs kill their process group on the
|
|
861
|
+
normal exit path too**, not only on timeout: a
|
|
862
|
+
backgrounded child with redirected stdio no longer
|
|
863
|
+
survives a passed evidence run. (The notifier runner was
|
|
864
|
+
checked and left alone: it does not start a session, so
|
|
865
|
+
no group exists to kill.)
|
|
866
|
+
- **P3-07 — the discovery `--version` probe is capped**
|
|
867
|
+
(64 KB retained per stream, child killed past the cap)
|
|
868
|
+
and honours its never-raises contract end to end.
|
|
869
|
+
- **P3-08 — herdr detection fails closed on malformed CLI
|
|
870
|
+
output**: null, non-object and deeply nested JSON all go
|
|
871
|
+
inert with the reason recorded, never an exception.
|
|
872
|
+
- **P3-09 — a named repo must exist.** An explicitly empty
|
|
873
|
+
or nonexistent/non-directory `--repo` is a clear exit-2
|
|
874
|
+
error in both patch and git modes (patch mode previously
|
|
875
|
+
accepted it and silently skipped policy, AGENTS.md and
|
|
876
|
+
evidence context); the MCP run_gate applies the same
|
|
877
|
+
validation.
|
|
878
|
+
- **P3-10 — the MCP transport enforces the protocol it
|
|
879
|
+
speaks**: frames whose `jsonrpc` field is absent or not
|
|
880
|
+
"2.0" get -32600 (notifications keep their no-response
|
|
881
|
+
semantics), non-finite ids are answered as null, and
|
|
882
|
+
responses are serialised with allow_nan=False.
|
|
883
|
+
- **P3-11 — one shared containment helper**: `_is_within`
|
|
884
|
+
lives in core and is used by the CLI loaders, the
|
|
885
|
+
file-context collector and the AGENTS.md collector
|
|
886
|
+
(previously two sites called commonpath directly, one
|
|
887
|
+
without the fail-closed ValueError handling).
|
|
888
|
+
- **P3-12 — notifier summaries are sanitised at the single
|
|
889
|
+
chokepoint** argv and the record share: control/escape
|
|
890
|
+
bytes are stripped (ANSI/OSC sequences lose their bytes)
|
|
891
|
+
and the summary is capped at 500 chars with an ellipsis
|
|
892
|
+
marker.
|
|
893
|
+
|
|
894
|
+
Follow-up completions from the tester's response to the
|
|
895
|
+
Phase 2/3 follow-ups (2026-11-06), each independently
|
|
896
|
+
re-verified against master before landing:
|
|
897
|
+
|
|
898
|
+
- **P3-02 follow-up — the 16K scan cap is removed.** The
|
|
899
|
+
prefix cap above blinded every rule past 16,384 chars
|
|
900
|
+
(a fake AWS key planted at offset 20,000 of one line
|
|
901
|
+
produced zero findings). The bounded regex gaps did
|
|
902
|
+
the DoS work — a ~1 MB pathological line scans in ~3 s,
|
|
903
|
+
near-linear — so lines are now scanned IN FULL up to a
|
|
904
|
+
1,000,000-char hard bound. Past the bound the prefix is
|
|
905
|
+
scanned and a minor `scan-line-truncated` finding
|
|
906
|
+
records that the tail was not checked: a line that long
|
|
907
|
+
never silently passes. (The residual that stands: SQL
|
|
908
|
+
keywords and their concatenation markers more than 200
|
|
909
|
+
chars apart are not one statement construction for the
|
|
910
|
+
`sql-string-build` rule.)
|
|
911
|
+
- **P3-04 follow-up — the sibling-process residual is
|
|
912
|
+
documented and surfaced, not closed.** `PR_SET_DUMPABLE`
|
|
913
|
+
protects only the gate process; in env-only mode a
|
|
914
|
+
sandboxed child can still read any dumpable same-uid
|
|
915
|
+
sibling process's environment via `/proc/<pid>/environ`
|
|
916
|
+
(shell, IDE, other agents). THREAT-MODEL gains residual
|
|
917
|
+
7 (the "provider keys never reach an evidence process"
|
|
918
|
+
property depends on namespace mode; env-only is not
|
|
919
|
+
suitable for untrusted trees), the README sandbox
|
|
920
|
+
paragraph carries the same qualification, and a run
|
|
921
|
+
that executes env-only while the gate environment
|
|
922
|
+
carries key-shaped variable names warns once on stderr
|
|
923
|
+
and records the exposure in the evidence record's
|
|
924
|
+
sandbox section (variable names only, never values).
|
|
925
|
+
- **P3-01 follow-up — MCP verdicts carry policy
|
|
926
|
+
visibility.** `run_gate` captures the CLI's stderr and
|
|
927
|
+
discards it on success, so the discovered-policy
|
|
928
|
+
warning never reached MCP clients. The verdict payload
|
|
929
|
+
(both `run_gate` and `get_verdict`) now carries
|
|
930
|
+
`policy_origin` and `policy_self_modification` from the
|
|
931
|
+
record — always present (null / [] when the run had no
|
|
932
|
+
policy section), so clients can rely on the shape.
|
|
933
|
+
|
|
934
|
+
### 0.13.0.dev0 (external Phase 2 test fixes)
|
|
935
|
+
|
|
936
|
+
Fixes from the external tester's Phase 2 report (2026-11-06),
|
|
937
|
+
each independently verified against master before landing:
|
|
938
|
+
|
|
939
|
+
- **P2-01 — strict + incomplete no longer downgrades a
|
|
940
|
+
blocker.** The `_gate` incomplete-coverage branch returned
|
|
941
|
+
"inconclusive" under strict mode even with a blocking
|
|
942
|
+
finding present; it now carries the same `not blocking`
|
|
943
|
+
guard as the degraded branch.
|
|
944
|
+
- **P2-02 — merge_findings is order-invariant.** The greedy
|
|
945
|
+
merge compared each finding only against each group's
|
|
946
|
+
founding title, so grouping (and, under `balanced`, the
|
|
947
|
+
verdict) depended on input order. Inputs are first sorted
|
|
948
|
+
by a canonical content key (file, category, line, title,
|
|
949
|
+
severity, sources — never the assignment-order id).
|
|
950
|
+
- **P2-03 — diversity family tags are normalised.**
|
|
951
|
+
`known_families` / `assess_panel` compare strip+casefold
|
|
952
|
+
forms, so "OpenAI" vs "openai" no longer counts as two
|
|
953
|
+
families and builder exclusion no longer fails silently
|
|
954
|
+
on case. Tags stored elsewhere are untouched.
|
|
955
|
+
- **P2-04 — policy loader robustness.** A UTF-8 BOM no
|
|
956
|
+
longer makes a valid policy unloadable (decoded as
|
|
957
|
+
utf-8-sig), and a deeply nested document now raises
|
|
958
|
+
PolicyError naming the excessive nesting depth instead of
|
|
959
|
+
leaking RecursionError.
|
|
960
|
+
- **P2-05 — risk path patterns component-anchored.** The
|
|
961
|
+
migrations, secrets and deserialization patterns no
|
|
962
|
+
longer match keywords buried inside longer components
|
|
963
|
+
("im**migration**", "no**secrets**.py"); canonical paths
|
|
964
|
+
(db/migrations/…, config/secrets.yaml, .env,
|
|
965
|
+
lib/deserialize.py) score exactly as before.
|
|
966
|
+
- **P2-06 — claimed outcomes read counts and negation.**
|
|
967
|
+
"12 passed, 0 failed" and "did not fail" are pass claims
|
|
968
|
+
(previously read as FAILED); "0 errors" is a pass claim
|
|
969
|
+
(previously no claim). Counts are read first, then a
|
|
970
|
+
negation guard, then the original word search.
|
|
971
|
+
- **P2-07 — claim parsing is linear and single-pass.** The
|
|
972
|
+
labelled-line trailing-outcome regex backtracked
|
|
973
|
+
quadratically on long separator runs (~12s at 12KB; now
|
|
974
|
+
milliseconds), and `verify_report_claims` no longer
|
|
975
|
+
parses the report text twice.
|
|
976
|
+
- **Survivor pins.** Regression tests now hold behaviours
|
|
977
|
+
the Phase 2 mutation pass verified: the balanced
|
|
978
|
+
unattributed-source guard, deterministic source-lens
|
|
979
|
+
stamping, tiering missing-seat dedup, and lens-pin edges
|
|
980
|
+
(ghost seat / unknown lens ignored, single-seat fallback,
|
|
981
|
+
dual pins).
|
|
982
|
+
|
|
983
|
+
**P2-08 — NOT landed (blocked, decision pending).** The
|
|
984
|
+
verified finding stands: under `balanced`, an unbound panel
|
|
985
|
+
gets positional lens assignment and the same finding can
|
|
986
|
+
flip PASS/BLOCKED by binding state. The prescribed
|
|
987
|
+
fail-closed fix (run_review raises without a seat_lenses
|
|
988
|
+
binding) cannot land as specified: the CLI and policy
|
|
989
|
+
surfaces have NO way to supply a binding (zero seat_lenses
|
|
990
|
+
references in cli.py / policy.py / userconfig.py — bindings
|
|
991
|
+
are produced only by bench's derive_seat_lenses and direct
|
|
992
|
+
library callers), so the raise would make the shipped
|
|
993
|
+
`--gate-profile balanced` flag and a policy-file
|
|
994
|
+
`[gate] profile = "balanced"` fail on every run with no
|
|
995
|
+
possible remedy. Landing it needs a product decision:
|
|
996
|
+
add a CLI/policy binding surface in the same change, or
|
|
997
|
+
have the CLI derive and record an explicit binding.
|
|
998
|
+
|
|
999
|
+
### 0.13.0.dev0 (external Phase 0–1 test fixes)
|
|
1000
|
+
|
|
1001
|
+
Version bumped to 0.13.0.dev0 (pyproject.toml and
|
|
1002
|
+
core.VERSION, kept identical) for the fixes from the
|
|
1003
|
+
external tester's Phase 0–1 report (2026-11-06):
|
|
1004
|
+
|
|
1005
|
+
- **D-001 — deterministic records.** run_review's job
|
|
1006
|
+
executor folded reviewer results in completion order, so
|
|
1007
|
+
reviewer_reports order, merged findings' sources order and
|
|
1008
|
+
the markdown render varied run to run for identical inputs.
|
|
1009
|
+
Results are now collected and folded in submission order
|
|
1010
|
+
(the run_verification pattern); gate semantics unchanged.
|
|
1011
|
+
- **D-003 — self-testing sdist.** MANIFEST.in now ships the
|
|
1012
|
+
full tests tree (fixtures included), probes/, bench/corpus/,
|
|
1013
|
+
examples/ and the top-level CHANGELOG.md, precedents.json,
|
|
1014
|
+
fixtures-manifest.json and THREAT-MODEL.md, so an unpacked
|
|
1015
|
+
sdist can run its own suite. bench/ beyond the corpus —
|
|
1016
|
+
registered run material — stays out.
|
|
1017
|
+
- **D-004 — `--version`.** The CLI answers `--version` with
|
|
1018
|
+
the package version and exit 0 (both entry points are the
|
|
1019
|
+
same main()).
|
|
1020
|
+
- **D-005 — subcommand `--help`.** `policy --help`,
|
|
1021
|
+
`bench --help` and `mcp --help` exited 2 with a bare usage
|
|
1022
|
+
line; all fifteen subcommands' `--help` now prints real
|
|
1023
|
+
help and exits 0.
|
|
1024
|
+
- **D-007 — environment-dependent tests skip cleanly.** The
|
|
1025
|
+
three test files needing the swarm's llm_chain module skip
|
|
1026
|
+
with a stated reason when it is not importable, instead of
|
|
1027
|
+
erroring at import.
|
|
1028
|
+
- **Flake fix.** The notify/risk "never changes the outcome"
|
|
1029
|
+
comparisons now also exclude the report's top-level
|
|
1030
|
+
duration_ms, which intermittently failed them on timing
|
|
1031
|
+
alone.
|
|
1032
|
+
|
|
1033
|
+
### verdict record schema v1.0.0
|
|
1034
|
+
|
|
1035
|
+
The run record is now a formal, versioned contract
|
|
1036
|
+
(`src/codereview/record_schema.py`): `RECORD_SCHEMA` +
|
|
1037
|
+
a stdlib-only validator, with every record validated by
|
|
1038
|
+
the core at emission and re-validated by `write_report`
|
|
1039
|
+
before it reaches disk — so the CLI, MCP, GitHub, and loop
|
|
1040
|
+
surfaces all emit the same v1 record. Added sections:
|
|
1041
|
+
`schema_version` ("1.0.0", alongside the unchanged
|
|
1042
|
+
`codereview.report/1` tag), uppercase `verdict` beside the
|
|
1043
|
+
legacy lowercase `gate`, `scope`, `checks` (run / not_run /
|
|
1044
|
+
unavailable — checks that did not run are visible),
|
|
1045
|
+
`panel` (seats, families, degraded state), `cost`
|
|
1046
|
+
(explicitly `not_available`: the core has no cost data and
|
|
1047
|
+
invents none), `duration_ms`, `waivers`, `provenance`,
|
|
1048
|
+
`record_identity` (content-hash id; writer-stamped file
|
|
1049
|
+
name), and `verdict_semantics` (a PASS means no actionable
|
|
1050
|
+
issue found under the checks that ran — not a correctness
|
|
1051
|
+
guarantee). Emission fixes, no verdict-logic changes:
|
|
1052
|
+
verify-mode records now carry `degraded_causes` and
|
|
1053
|
+
`patch_chars` (both previously absent there). Documented
|
|
1054
|
+
gaps, stated in the schema and README: verify mode lists
|
|
1055
|
+
coverage/evidence/policy as `checks.not_run`; the loop
|
|
1056
|
+
summary is a container, not a v1 verdict record; legacy
|
|
1057
|
+
records without `schema_version` are pre-v1, not corrupt.
|
|
1058
|
+
Golden PASS / BLOCKED / INCONCLUSIVE fixtures and parity,
|
|
1059
|
+
tamper, and self-tests in `tests/test_record_schema.py`.
|
|
1060
|
+
|
|
1061
|
+
### Epic E2 + E4 (tiered panels + escalation)
|
|
1062
|
+
|
|
1063
|
+
E1's risk band now sizes the panel that reviews a change,
|
|
1064
|
+
and the smaller tiers carry a safety valve: escalation to
|
|
1065
|
+
the full panel within the same run. Both are driven by the
|
|
1066
|
+
[risk] table of .codereview.toml (`tiering = true`, which
|
|
1067
|
+
requires `enabled = true` — policy load rejects the
|
|
1068
|
+
combination otherwise, in the B1 style) and recorded in
|
|
1069
|
+
the run record's new `tiering` section
|
|
1070
|
+
(`codereview.tiering/1`). Unconfigured, a run is
|
|
1071
|
+
byte-identical to before this epic.
|
|
1072
|
+
|
|
1073
|
+
- **Tiering (E2).** Band "low" runs a designated
|
|
1074
|
+
pre-screen seat (or seats) only (`prescreen_seats`;
|
|
1075
|
+
default: the composed panel's first seat); "medium"
|
|
1076
|
+
runs a reduced panel (`medium_seats`; default: the
|
|
1077
|
+
first tasked seat per lens); "high" runs the full
|
|
1078
|
+
panel and requires evidence — a supplied evidence
|
|
1079
|
+
record is judged by the Epic C rule exactly as today,
|
|
1080
|
+
and a missing one blocks the gate under C3's
|
|
1081
|
+
missing-required-evidence consequence. Under an
|
|
1082
|
+
explicit seat->lens binding the tier is a subset of the
|
|
1083
|
+
bound tasks — a seat serves exactly its bound lens(es),
|
|
1084
|
+
never a re-mapping (S1); under the legacy assignment
|
|
1085
|
+
the tier's seats form a smaller panel tasked by the
|
|
1086
|
+
ordinary derivation rule. A configured seat that is not
|
|
1087
|
+
on the panel is recorded as missing, never silently
|
|
1088
|
+
dropped; if a tier would be left with no serving seat,
|
|
1089
|
+
the run falls back to the full panel with the reason
|
|
1090
|
+
recorded.
|
|
1091
|
+
- **Escalation (E4).** The reduced tiers execute as
|
|
1092
|
+
stage 1; if stage 1 returns a blocking finding (the
|
|
1093
|
+
gate's own blocking severity set), a reviewer failure,
|
|
1094
|
+
or leaves a lens the full panel tasks unserved, the
|
|
1095
|
+
rest of the panel executes as stage 2 in the same run
|
|
1096
|
+
and the verdict is computed on the full evidence by the
|
|
1097
|
+
normal rules. Escalation never removes a stage-1
|
|
1098
|
+
report or finding. Every trigger that fired is
|
|
1099
|
+
recorded in the `tiering` section and summarised in
|
|
1100
|
+
the gate reasons.
|
|
1101
|
+
- **Cost.** On the public bench split (27 cases, a
|
|
1102
|
+
synthetic all-pass 4-seat panel, default thresholds:
|
|
1103
|
+
26 cases band low, 1 medium, 0 high) tiering spends 54
|
|
1104
|
+
seat calls against the untiered panel's 108 — a 50%
|
|
1105
|
+
drop. High-band recall is unchanged by construction:
|
|
1106
|
+
the high tier's stage is the identical full task list
|
|
1107
|
+
and prompts as an untiered run (fixture-pinned), plus
|
|
1108
|
+
the evidence requirement.
|
|
1109
|
+
### 2026-11-05 (registration verifier: Addendum A append)
|
|
1110
|
+
|
|
1111
|
+
Addendum A to the 2026-11-05 pre-registration was appended to the
|
|
1112
|
+
registration's §3 on 2026-11-05 (signed the same day by the owner
|
|
1113
|
+
and one independent reviewer; its item-2 disclosure completed and
|
|
1114
|
+
its item-4 wording aligned to the merged empty-content
|
|
1115
|
+
classification; addendum body hash
|
|
1116
|
+
`d83c93977d66769c614f6510159c0713f3b2addf8ba3c7854a7d6b5bdeea528d`,
|
|
1117
|
+
recorded in §3). The append changed the registration's body hash,
|
|
1118
|
+
so `bench/scripts/verify_registration.py` now treats the
|
|
1119
|
+
Baseline-v2-§6 post-fill body state the way it already treated the
|
|
1120
|
+
A5 set hash:
|
|
1121
|
+
|
|
1122
|
+
- The post-fill body hash (`9e38217e...`) is reported SUPERSEDED,
|
|
1123
|
+
and only when the body recomputes to the new state; the
|
|
1124
|
+
post-append body hash (`5d9bea16...`) is the current
|
|
1125
|
+
expectation, verified by strict equality. No check was removed
|
|
1126
|
+
or weakened: exit 0 still requires the registration body to hash
|
|
1127
|
+
to exactly the one authorised current value, and any other body
|
|
1128
|
+
state — including the superseded one — fails. Attempts consumed
|
|
1129
|
+
under the registration remain 0; no run was started.
|
|
1130
|
+
|
|
1131
|
+
### Epic C5 (sandbox safety audit)
|
|
1132
|
+
|
|
1133
|
+
Ticket C5's machinery largely shipped with the Epic C runner;
|
|
1134
|
+
this track audited it line by line against the ticket ("no
|
|
1135
|
+
network by default, resource limits, run in temp worktree;
|
|
1136
|
+
never run untrusted PR code with secrets" — acceptance:
|
|
1137
|
+
documented threat model + tests) and closed the one real gap.
|
|
1138
|
+
|
|
1139
|
+
- **Gap closed — captured output is now capped.** The
|
|
1140
|
+
executor buffered a child's full stdout+stderr in the gate
|
|
1141
|
+
process's memory (`communicate()`), so a spewing command
|
|
1142
|
+
could grow it without bound until the timeout fired. Each
|
|
1143
|
+
stream is now drained by a reader thread under an 8 MiB
|
|
1144
|
+
per-stream cap: past the cap the parent keeps draining and
|
|
1145
|
+
discarding (the child never blocks on a full pipe and its
|
|
1146
|
+
exit status is unaffected), the record's hash and tail
|
|
1147
|
+
cover the capped bytes, and a truncation marker in the
|
|
1148
|
+
output says when the cap bit; and if a detached process
|
|
1149
|
+
holds a pipe open past the post-kill settle, the capture is
|
|
1150
|
+
marked incomplete rather than silently partial (gate
|
|
1151
|
+
round-1 majors, accepted in substance). The record's
|
|
1152
|
+
`sandbox.limits` block states the cap. The wait lives in a named supervised
|
|
1153
|
+
wait (`_supervise_sandbox_run`) — deadline-bounded waits,
|
|
1154
|
+
bounded joins, process-group kill on expiry — and the v14
|
|
1155
|
+
subprocess-timeout guard accepts it by name, following the
|
|
1156
|
+
Epic K2 seat-wait precedent.
|
|
1157
|
+
- **Already met, now pinned** (`tests/test_sandbox_c5.py`):
|
|
1158
|
+
the CPU and per-file rlimits actually reach the child;
|
|
1159
|
+
provider-key-shaped variables, tokens, and the SSH agent
|
|
1160
|
+
socket never cross the allow-list, and `allow_network`
|
|
1161
|
+
re-admits proxy variables only; the throwaway copy is
|
|
1162
|
+
deleted on every exit path (normal, timeout kill, copy
|
|
1163
|
+
failure, claim verification); namespace mode is a real,
|
|
1164
|
+
different network namespace in which a connect attempt
|
|
1165
|
+
fails (where `unshare` works).
|
|
1166
|
+
- **Documented, not faked:** `THREAT-MODEL.md` at the
|
|
1167
|
+
repository root is the written threat model — controls,
|
|
1168
|
+
trust boundaries, and residual risks stated plainly. The
|
|
1169
|
+
headline residuals: no filesystem isolation (absolute-path
|
|
1170
|
+
reads/writes escape the copy — run the CLI in a VM or
|
|
1171
|
+
container for genuinely hostile trees), and **no memory
|
|
1172
|
+
cap**: an address-space rlimit would break legitimate
|
|
1173
|
+
toolchains (Node/V8 and JVM address-space reservations;
|
|
1174
|
+
`npm` is a built-in adapter), so memory growth is bounded
|
|
1175
|
+
only by the wall-clock/CPU limits and the risk is
|
|
1176
|
+
documented rather than theatre-capped.
|
|
1177
|
+
### Epic E1 (risk scorer)
|
|
1178
|
+
|
|
1179
|
+
A review run can now record a deterministic risk score for
|
|
1180
|
+
the change under review. When the project policy's new
|
|
1181
|
+
`[risk]` table sets `enabled = true`, the run record gains a
|
|
1182
|
+
`risk` section: the score (0–100), its band (`low` /
|
|
1183
|
+
`medium` / `high`), a human-readable reason for every point,
|
|
1184
|
+
and the per-signal breakdown (files and lines changed,
|
|
1185
|
+
pattern classes touched, dependency files).
|
|
1186
|
+
|
|
1187
|
+
- **Deterministic by contract:** `codereview.risk` is a pure
|
|
1188
|
+
function of the kept diff sections and the resolved
|
|
1189
|
+
thresholds — no model calls, no randomness, no clock. The
|
|
1190
|
+
same diff under the same policy scores identically on
|
|
1191
|
+
every run (test-pinned, including section order).
|
|
1192
|
+
- **Signals:** touched sensitive patterns — auth, payments,
|
|
1193
|
+
crypto, migrations, secrets, deserialization — matched on
|
|
1194
|
+
changed paths and added lines, each class contributing its
|
|
1195
|
+
points once with the pattern subtotal capped at 60;
|
|
1196
|
+
dependency-manifest changes (+15: package/requirements/
|
|
1197
|
+
cargo/go/composer/gem manifests and lockfiles, `*.txt`
|
|
1198
|
+
under a `requirements/` directory included); change size
|
|
1199
|
+
in fixed tiers up to +25.
|
|
1200
|
+
- **Descriptive only:** in E1 the score never changes a
|
|
1201
|
+
verdict, a finding, or any gate input (test-compared with
|
|
1202
|
+
risk on vs off). Tiering (E2) and escalation (E4) consume
|
|
1203
|
+
the section next.
|
|
1204
|
+
- **Schema:** `[risk]` (`enabled`, `medium_threshold`,
|
|
1205
|
+
`high_threshold`) is validated by the policy loader with
|
|
1206
|
+
line-referenced errors like every other table — non-bool
|
|
1207
|
+
`enabled`, thresholds outside 0–100, and an inverted
|
|
1208
|
+
medium/high pair are loud errors. `codereview policy
|
|
1209
|
+
check` renders the table when — and only when — scoring
|
|
1210
|
+
is enabled.
|
|
1211
|
+
- **Equivalence:** with `[risk]` absent or disabled, a run
|
|
1212
|
+
is unchanged in every particular — no `risk` key in the
|
|
1213
|
+
record; the registered bench sets are untouched.
|
|
1214
|
+
- **Verification:** 38 new tests (`tests/test_risk.py`),
|
|
1215
|
+
37 failing on pre-E1 master (the module does not exist
|
|
1216
|
+
there; the one pass is an unknown-table coincidence in a
|
|
1217
|
+
negative-validation test) — scorer fixtures per pattern
|
|
1218
|
+
class, size tiers, the pattern cap, determinism, policy
|
|
1219
|
+
schema errors, and record integration through
|
|
1220
|
+
`run_review` with stub seats.
|
|
1221
|
+
|
|
1222
|
+
### Epic E3 (panel-diversity enforcement)
|
|
1223
|
+
|
|
1224
|
+
A project can now enforce panel diversity in the
|
|
1225
|
+
`[diversity]` table of its `.codereview.toml`, judged on the
|
|
1226
|
+
COMPOSED panel before any seat is called:
|
|
1227
|
+
|
|
1228
|
+
- **Minimum families:** `min_families = N` requires the panel
|
|
1229
|
+
to span at least N distinct known vendor families (seat
|
|
1230
|
+
metadata — registry tags or `--seat-meta`; `unknown` never
|
|
1231
|
+
counts). This is distinct from `[panel] min_families`,
|
|
1232
|
+
which judges the seats that actually served, after the
|
|
1233
|
+
round. A violation labels the run degraded with the cause
|
|
1234
|
+
`panel diversity below minimum` and is recorded in the
|
|
1235
|
+
record's `diversity` section and as an applied
|
|
1236
|
+
`diversity.min_families` rule in the policy section.
|
|
1237
|
+
- **Enforcement mode:** `enforce = "degrade"` (the default)
|
|
1238
|
+
records the violation as advisory — never gate-changing.
|
|
1239
|
+
`enforce = "block"` additionally makes it a blocking
|
|
1240
|
+
policy violation, exactly like any other.
|
|
1241
|
+
- **Builder exclusion:** with `exclude_builder_family =
|
|
1242
|
+
true`, when the run's caller knows the builder's vendor
|
|
1243
|
+
family (the `builder_family` argument — no CLI run path
|
|
1244
|
+
knows it today, so there the exclusion simply never
|
|
1245
|
+
engages; no builder identity is ever invented), seats of
|
|
1246
|
+
that family are excluded from the panel when the remaining
|
|
1247
|
+
panel still meets the minimum — otherwise they stay, and
|
|
1248
|
+
the record says why. An excluded seat's lens is never
|
|
1249
|
+
reassigned by the check: the seat->lens binding machinery
|
|
1250
|
+
treats it as an absent seat.
|
|
1251
|
+
- **Equivalence:** with no `[diversity]` configuration, a
|
|
1252
|
+
run is unchanged in every particular — no `diversity` key
|
|
1253
|
+
in the record, no applied rule, identical verdicts; the
|
|
1254
|
+
registered bench sets are untouched.
|
|
1255
|
+
- **Verification:** 27 new tests (`tests/test_diversity.py`)
|
|
1256
|
+
— panel assessment (minimum met/breached, unknown-family
|
|
1257
|
+
handling, builder exclusion possible/refused/emptying/
|
|
1258
|
+
unknown/absent), policy schema + rendering, and run_review
|
|
1259
|
+
integration (degrade vs block, exclusion never calling the
|
|
1260
|
+
builder's seats, unconfigured equivalence).
|
|
1261
|
+
|
|
1262
|
+
### Epic C4 (builder-claim verification in the run record)
|
|
1263
|
+
|
|
1264
|
+
Builder-claim verification now lands in the run record. The
|
|
1265
|
+
machinery is not new — `evidence.parse_claims` /
|
|
1266
|
+
`evidence.verify_claims` (Epic C) already parsed a builder
|
|
1267
|
+
work report's claimed commands and re-ran them in the sandbox
|
|
1268
|
+
behind the standalone `codereview evidence --claims` surface.
|
|
1269
|
+
What is new is the record-facing contract around them
|
|
1270
|
+
(`src/codereview/claims.py`, composing the evidence runner —
|
|
1271
|
+
no second parser, no second executor):
|
|
1272
|
+
|
|
1273
|
+
- **Recorded in the run record:** on a verification run
|
|
1274
|
+
(`--verify-previous` with `--builder-response`), the record
|
|
1275
|
+
now carries a `claim_verification` section: per-claim
|
|
1276
|
+
entries with the claimed outcome, the re-run's actual exit
|
|
1277
|
+
code, duration, and output sha256, plus a summary. A run
|
|
1278
|
+
with no builder report carries no section.
|
|
1279
|
+
- **Statuses:** `verified` (the re-run matches the claim),
|
|
1280
|
+
`mismatch` (the re-run contradicts it — the claim is
|
|
1281
|
+
downgraded, recorded as not standing with claimed vs
|
|
1282
|
+
actual outcomes stated), `unverifiable` (the re-run could
|
|
1283
|
+
not be performed), and `unparsed` (a line shaped like a
|
|
1284
|
+
claim — a backticked command or a claim label — that does
|
|
1285
|
+
not parse to an unambiguous command + outcome: recorded,
|
|
1286
|
+
never guessed at, never executed). Ordinary prose is not a
|
|
1287
|
+
claim attempt and is not recorded.
|
|
1288
|
+
- **Evidence, never a gate input:** claim verification is
|
|
1289
|
+
recorded evidence only. No claim status changes the
|
|
1290
|
+
verdict, a finding, or any other gate input, and nothing
|
|
1291
|
+
in the gate reads the section — a mismatch downgrades the
|
|
1292
|
+
claim, never the gate (test-compared: identical panel,
|
|
1293
|
+
honest vs downgraded claims, identical gate). The section
|
|
1294
|
+
states this boundary in its own `boundary` field.
|
|
1295
|
+
- **Never raises:** malformed reports, non-text input, a
|
|
1296
|
+
missing tree, and sandbox failures are recorded outcomes
|
|
1297
|
+
(`unverifiable`, with the reason), never exceptions; the
|
|
1298
|
+
run's record is produced regardless.
|
|
1299
|
+
- **Verification:** 20 new tests (`tests/test_claims.py`) —
|
|
1300
|
+
18 fail on pre-C4 master with an empty-module shim (the
|
|
1301
|
+
module does not exist there at all); the 2 passes pin
|
|
1302
|
+
pre-existing behaviour by design (no builder report ⇒ no
|
|
1303
|
+
section; a review run ⇒ no section).
|
|
1304
|
+
|
|
1305
|
+
### Epic L3 (lifecycle hooks)
|
|
1306
|
+
|
|
1307
|
+
A project can now declare shell commands in the `[hooks]`
|
|
1308
|
+
table of its `.codereview.toml` that fire around a review
|
|
1309
|
+
run: `pre_run` before any seat call, `on_blocked` when the
|
|
1310
|
+
run's first seat ended in a K2 blocked class (quota / auth /
|
|
1311
|
+
input / stalled), and `post_verdict` on every run — the two
|
|
1312
|
+
post-record events after the record is written, so they are
|
|
1313
|
+
handed its path (as `$CODEREVIEW_HOOK_RECORD`, and as `$2`;
|
|
1314
|
+
the event name is `$CODEREVIEW_HOOK_EVENT` / `$1`).
|
|
1315
|
+
|
|
1316
|
+
- **One-way enforcement:** hooks are notification, never a
|
|
1317
|
+
gate input. Each hook's outcome — status, exit code,
|
|
1318
|
+
duration, output sha256 + redacted tail — is captured into
|
|
1319
|
+
the record's `hooks` section; a hook that fails, times
|
|
1320
|
+
out, or names a missing command is recorded and the run
|
|
1321
|
+
continues, its verdict exactly what it would have been
|
|
1322
|
+
without the hook (test-compared).
|
|
1323
|
+
- **Sandboxed, C5 verbatim:** hooks execute through the
|
|
1324
|
+
Epic C evidence runner (`codereview.hooks` builds a
|
|
1325
|
+
single-command evidence config — no second executor):
|
|
1326
|
+
throwaway copy of the tree as cwd, allow-listed
|
|
1327
|
+
environment (no provider keys), no network by default,
|
|
1328
|
+
per-hook timeout (the `[hooks] timeout` key, defaulting
|
|
1329
|
+
to the evidence bound) with process-group kill.
|
|
1330
|
+
- **Schema:** `[hooks]` is validated by the policy loader
|
|
1331
|
+
with line-referenced errors like every other table
|
|
1332
|
+
(unknown keys, empty/non-string commands, non-positive
|
|
1333
|
+
timeouts are loud errors). `codereview policy check`
|
|
1334
|
+
renders the table when — and only when — a hook is
|
|
1335
|
+
configured.
|
|
1336
|
+
- **Equivalence:** with no `[hooks]` commands, a run is
|
|
1337
|
+
unchanged in every particular — no `hooks` key in the
|
|
1338
|
+
record, identical prompts and verdicts; the registered
|
|
1339
|
+
bench sets are untouched.
|
|
1340
|
+
- **Verification:** 25 new tests (`tests/test_hooks.py`),
|
|
1341
|
+
none passing on pre-L3 master — schema errors, sandbox
|
|
1342
|
+
delivery (event/record/env), the C5 environment marker,
|
|
1343
|
+
exactly-once + ordering through the real CLI with stub
|
|
1344
|
+
seats, the fallback-primary trigger, and verdict identity
|
|
1345
|
+
across absent vs failing hooks.
|
|
1346
|
+
|
|
1347
|
+
### Epic L2 (MCP server for the gate)
|
|
1348
|
+
|
|
1349
|
+
The gate is now callable from inside another agent's loop:
|
|
1350
|
+
`codereview mcp serve` speaks the Model Context Protocol
|
|
1351
|
+
over stdio and exposes two tools — `run_gate` (a patch, or a
|
|
1352
|
+
git range in a repo, plus an optional policy path → the
|
|
1353
|
+
verdict and the run record's path) and `get_verdict` (a
|
|
1354
|
+
stored record → verdict summary and findings).
|
|
1355
|
+
|
|
1356
|
+
- **In-tree protocol subset, stdlib only:** the wire layer
|
|
1357
|
+
(`codereview.mcp_transport` — newline-delimited JSON-RPC
|
|
1358
|
+
2.0 framing, dispatch, error ledger) and the tool layer
|
|
1359
|
+
(`codereview.mcp_tools` — handshake negotiation for
|
|
1360
|
+
protocol versions 2025-03-26 / 2024-11-05, tools/list,
|
|
1361
|
+
tools/call) are separate modules; no new dependency.
|
|
1362
|
+
- **Serving only:** the server is a front door onto the
|
|
1363
|
+
existing gate, not a new gate — a run goes through the
|
|
1364
|
+
CLI's own `main()` with the arguments translated, so the
|
|
1365
|
+
served verdict is the CLI's verdict on the same change and
|
|
1366
|
+
the record on disk is the ordinary run record. The review
|
|
1367
|
+
path consumes no MCP tools; reviewers stay read-only and
|
|
1368
|
+
tool-less (Epic F2 unchanged).
|
|
1369
|
+
- **No credentials, no built-in panel:** the server stores
|
|
1370
|
+
nothing; seats resolve from the operator's saved setup (or
|
|
1371
|
+
$CODEREVIEW_PROVIDERS) under the CLI's own discipline.
|
|
1372
|
+
With no setup, run_gate answers with the CLI's not-set-up
|
|
1373
|
+
message verbatim.
|
|
1374
|
+
- **Fail-closed wire:** malformed JSON, non-object frames,
|
|
1375
|
+
oversized frames (16 MiB cap, discarded in sync), and
|
|
1376
|
+
handler crashes are each answered or recorded in the
|
|
1377
|
+
server's error ledger and logged to stderr — the loop
|
|
1378
|
+
never dies on a bad frame. Tool arguments are validated
|
|
1379
|
+
against the published schemas (unknown keys, wrong types,
|
|
1380
|
+
patch+range together are refused before anything runs).
|
|
1381
|
+
- **Verification:** 15 new tests (`tests/test_mcp.py`), none
|
|
1382
|
+
passing on pre-L2 master — transport failure modes,
|
|
1383
|
+
handshake/list, MCP-vs-CLI verdict equality in-process,
|
|
1384
|
+
the no-setup answer, and a full subprocess session (a
|
|
1385
|
+
minimal in-tree client over a real stdio pipe, seats from
|
|
1386
|
+
a local OpenAI-compatible stub) whose verdict is compared
|
|
1387
|
+
against the CLI binary's on the same diff. A brand-name
|
|
1388
|
+
MCP client session (Claude Code / Codex) is UNVERIFIED:
|
|
1389
|
+
no agent CLI on the build machine is logged in. Full
|
|
1390
|
+
suite 1005/1005 on the branch; 1018/1018 on master after
|
|
1391
|
+
the parallel L5 merge.
|
|
1392
|
+
### Epic L4 (attention notifications)
|
|
1393
|
+
|
|
1394
|
+
A gate run can now call the operator back: when a seat
|
|
1395
|
+
enters a blocked state (the Epic K2 classes) or the gate
|
|
1396
|
+
reaches a verdict, the run executes one local command the
|
|
1397
|
+
operator configured — a terminal bell, a desktop toast, a
|
|
1398
|
+
herdr-style attention hook. No hosted service, no
|
|
1399
|
+
CodeReview-managed endpoint.
|
|
1400
|
+
|
|
1401
|
+
- **User setup only:** the command lives in a `[notify]`
|
|
1402
|
+
table in the saved user config (`command = [...]`, an
|
|
1403
|
+
argv list) — notifications are operator preference, not
|
|
1404
|
+
project contract, so the project policy
|
|
1405
|
+
(.codereview.toml) has no notify key and refuses the
|
|
1406
|
+
table as unknown. The wizard's saved-file writer carries
|
|
1407
|
+
the table only when set; an unchanged setup stays
|
|
1408
|
+
byte-identical.
|
|
1409
|
+
- **Argv contract:** the command runs directly (no shell)
|
|
1410
|
+
with two arguments appended — the event name and a
|
|
1411
|
+
one-line summary (`seat cx blocked:quota`,
|
|
1412
|
+
`gate blocked: 3 finding(s)`). Duplicate suppression is
|
|
1413
|
+
per run: once per (seat, blocked class), once for the
|
|
1414
|
+
verdict.
|
|
1415
|
+
- **Bounded and recorded:** each invocation is capped at
|
|
1416
|
+
5 s; its outcome (ok / failed / timeout / error, exit
|
|
1417
|
+
code, duration) lands in the record's `notifications`
|
|
1418
|
+
section. A failing or hanging notifier never delays the
|
|
1419
|
+
run beyond the bound and never changes a finding or the
|
|
1420
|
+
verdict — test-compared against the unconfigured run.
|
|
1421
|
+
Unconfigured means zero trace: no invocations, no record
|
|
1422
|
+
section, prompts byte-identical.
|
|
1423
|
+
- **Distinct from Epic L3 hooks:** L3 is project-level
|
|
1424
|
+
automation in the policy file, executed in the evidence
|
|
1425
|
+
sandbox; L4 is the operator's personal command from the
|
|
1426
|
+
user setup, executed as the operator. The two share no
|
|
1427
|
+
machinery.
|
|
1428
|
+
- **Verification:** 28 new tests (`tests/test_notify.py`);
|
|
1429
|
+
23 fail on pre-L4 master, the 5 passes being the
|
|
1430
|
+
by-design pins (policy rejection, byte-identical saved
|
|
1431
|
+
file, and the unset-means-silent equivalences). Coverage:
|
|
1432
|
+
the Notifier unit contract, the real subprocess runner
|
|
1433
|
+
(including a hanging command bounded by the timeout),
|
|
1434
|
+
the user-config table (round-trip, byte-identical
|
|
1435
|
+
default, loud ValueError when malformed), run_review and
|
|
1436
|
+
run_verification integration (exactly-once events,
|
|
1437
|
+
outcome-invariance, prompt identity), and the CLI seam
|
|
1438
|
+
end to end (setup → run → record on disk).
|
|
1439
|
+
|
|
1440
|
+
### Epic L5 (first-class local-model seats)
|
|
1441
|
+
|
|
1442
|
+
Local model servers are now first-class seats: named
|
|
1443
|
+
presets in the provider registry for the endpoints a user
|
|
1444
|
+
runs on their own machine, offered by the setup wizard with
|
|
1445
|
+
a short recipe, replacing the unnamed `openai-compatible:`
|
|
1446
|
+
escape hatch for the local case (the hatch itself is
|
|
1447
|
+
unchanged).
|
|
1448
|
+
|
|
1449
|
+
- **Registry** (`codereview.registry`): `ollama`
|
|
1450
|
+
(http://localhost:11434/v1) and `lmstudio`
|
|
1451
|
+
(http://localhost:1234/v1) presets with family tag
|
|
1452
|
+
`local`, plus the generic form
|
|
1453
|
+
`local:<name>:<base-url>|<model>` for any other
|
|
1454
|
+
OpenAI-compatible local endpoint. A preset names an
|
|
1455
|
+
endpoint, never a panel — the serving model is still the
|
|
1456
|
+
user's pick (`ollama:<model>`), per the first-run rule.
|
|
1457
|
+
- **Keyless by construction:** local presets carry
|
|
1458
|
+
`key_env` None — no credential is read, no Authorization
|
|
1459
|
+
header is sent, and the wizard never prompts for a key
|
|
1460
|
+
for a local seat (there is none). Account rotation does
|
|
1461
|
+
not invent `KEY_ENV_2`-style names for them; the account
|
|
1462
|
+
label in records is `local:<seat>`.
|
|
1463
|
+
- **Wizard:** local presets are listed as their own
|
|
1464
|
+
section — endpoint, suggested model, and the recipe
|
|
1465
|
+
(start the server → pick the preset → `codereview
|
|
1466
|
+
doctor` probe) — and the per-seat summary states the
|
|
1467
|
+
no-network nature ("runs on your machine; the review
|
|
1468
|
+
never leaves it"). Hosted seats' key-env summary is
|
|
1469
|
+
unchanged.
|
|
1470
|
+
- **Ordinary seats:** a local seat is built, pooled, probed
|
|
1471
|
+
by `doctor`, and recorded like any other, with family
|
|
1472
|
+
`local` travelling on the callable into the record's
|
|
1473
|
+
seat metadata — so the independence checks see it
|
|
1474
|
+
honestly (never a second vendor family). Non-local specs
|
|
1475
|
+
and records are untouched. 13 new tests
|
|
1476
|
+
(`tests/test_local_seats.py`), 11 failing on pre-L5
|
|
1477
|
+
master (the escape-hatch regression and the
|
|
1478
|
+
malformed-spec test pass on both by design); full suite
|
|
1479
|
+
1003/1003.
|
|
1480
|
+
|
|
1481
|
+
### Epic L1 (AGENTS.md ingestion)
|
|
1482
|
+
|
|
1483
|
+
Reviewer context now includes the repository's own guidance:
|
|
1484
|
+
`AGENTS.md` at the repo root and nested along the changed
|
|
1485
|
+
paths is ingested at run start, alongside (never instead of)
|
|
1486
|
+
`.codereview.toml`.
|
|
1487
|
+
|
|
1488
|
+
- **Ingestion** (`codereview.agentsmd`, wired into repo
|
|
1489
|
+
reviews; `--no-agents-context` opts out): discovery is
|
|
1490
|
+
deterministic — the root plus the ancestor directories of
|
|
1491
|
+
the changed paths, ordered nearest-wins (root first; the
|
|
1492
|
+
file nearest a changed file governs on conflict, and the
|
|
1493
|
+
prompt says so). A `## Code Review Rules` section is
|
|
1494
|
+
surfaced at the head of the attached context as advisory
|
|
1495
|
+
review rules the panel may cite in findings.
|
|
1496
|
+
- **Trust model:** the files are untrusted content on the
|
|
1497
|
+
diff's own terms — passed through the review path's
|
|
1498
|
+
redaction, delimited in the prompt as data under a header
|
|
1499
|
+
that forbids following instructions inside them, and
|
|
1500
|
+
structurally unable to change a gate rule, the panel, or
|
|
1501
|
+
a record field (pinned by a prompt-injection test: a
|
|
1502
|
+
hostile AGENTS.md demanding a pass leaves the gate, the
|
|
1503
|
+
findings, the panel, and the gate-profile section
|
|
1504
|
+
byte-identical to the no-guidance control run). The TOML
|
|
1505
|
+
policy remains the only machine-enforced layer.
|
|
1506
|
+
- **Bounds + provenance:** capped per file (16,000 chars)
|
|
1507
|
+
and in total (32,000), truncation marked in the text and
|
|
1508
|
+
recorded per file; the run record's `agents_context`
|
|
1509
|
+
section carries each ingested file's path and the sha256
|
|
1510
|
+
of its raw bytes, its sizes, and whether its rules section
|
|
1511
|
+
was present. Nothing is silently dropped — the collector's
|
|
1512
|
+
ledger covers skipped files with reasons.
|
|
1513
|
+
- **Equivalence:** with no AGENTS.md on any changed path
|
|
1514
|
+
(or with the opt-out), prompts and records are
|
|
1515
|
+
byte-identical to the pre-L1 construction; the bench and
|
|
1516
|
+
registered confirmation runs never pass the new argument,
|
|
1517
|
+
so their prompts are unchanged. 17 new tests
|
|
1518
|
+
(`tests/test_agents_md.py`), none passing on pre-L1
|
|
1519
|
+
master; full suite 990/990.
|
|
1520
|
+
|
|
1521
|
+
### Epic K3b (herdr ecosystem artefact)
|
|
1522
|
+
|
|
1523
|
+
Presence in herdr's ecosystem, with the engine staying in
|
|
1524
|
+
the pip package: a herdr plugin artefact and an agent skill,
|
|
1525
|
+
both shipped in this repo (and, via a new `MANIFEST.in`, in
|
|
1526
|
+
the sdist — verified: a test sdist build carries both).
|
|
1527
|
+
|
|
1528
|
+
- **Herdr plugin** (`herdr-plugin-codereview/`):
|
|
1529
|
+
`herdr-plugin.toml` (`superstartelecom.codereview`,
|
|
1530
|
+
`min_herdr_version` 0.9.0 — the floor K3a verified)
|
|
1531
|
+
declares one workspace action, "CodeReview: review current
|
|
1532
|
+
workspace", and one overlay pane entrypoint. Both run the
|
|
1533
|
+
launcher, `codereview_launch.py` (Python, stdlib only): it
|
|
1534
|
+
locates `codereview` on PATH — if absent it prints the
|
|
1535
|
+
`pip install codereview-ai` hint and exits non-zero
|
|
1536
|
+
*before doing anything else*, so that failure has no side
|
|
1537
|
+
effects — resolves the workspace from herdr's invocation
|
|
1538
|
+
context (`workspace_cwd`), reads an optional
|
|
1539
|
+
`policy`/`out` config from the plugin config dir, runs the
|
|
1540
|
+
gate in the workspace streaming its output, and exits with
|
|
1541
|
+
the gate's own exit code. The launcher stores no
|
|
1542
|
+
credentials, makes no network calls, and writes nothing
|
|
1543
|
+
itself (imports and read-only file opens are pinned by
|
|
1544
|
+
AST tests); the only files a run creates are CodeReview's
|
|
1545
|
+
own records in the reviewed workspace.
|
|
1546
|
+
- **Agent skill** (`skills/codereview/SKILL.md`, standard
|
|
1547
|
+
Agent Skills format): teaches an agent to run the gate,
|
|
1548
|
+
read the JSON record as the source of truth, and report
|
|
1549
|
+
PASS / BLOCKED / INCONCLUSIVE honestly — a BLOCKED is
|
|
1550
|
+
never softened, an INCONCLUSIVE is never upgraded, and no
|
|
1551
|
+
verdict may be claimed that the record doesn't carry.
|
|
1552
|
+
- **Verified here:** the launcher against a stub
|
|
1553
|
+
`codereview` honouring the real CLI's printed contract
|
|
1554
|
+
(verdict/record lines, exit codes 0/1/2, config
|
|
1555
|
+
pass-through, no-side-effect failure); the plugin's
|
|
1556
|
+
action contract against a stub `herdr` invoking the
|
|
1557
|
+
manifest argv the way herdr does (cwd = plugin dir,
|
|
1558
|
+
`HERDR_PLUGIN_*` env); and an end-to-end smoke of the
|
|
1559
|
+
launcher driving the real CLI (two scripted agent seats)
|
|
1560
|
+
to a real PASS record on a scratch repo. 20 new tests,
|
|
1561
|
+
all failing on pre-K3b master.
|
|
1562
|
+
- **UNVERIFIED — plugin link.** Real herdr is not installed
|
|
1563
|
+
on the build machine (no Rust toolchain), so
|
|
1564
|
+
`herdr plugin link` on a clean profile and the action's
|
|
1565
|
+
appearance in `herdr plugin action list` on a real herdr
|
|
1566
|
+
have NOT been run; the stub-herdr contract test above is
|
|
1567
|
+
the closest honest exercise. The marketplace listing
|
|
1568
|
+
remains GitHub-gated and out of scope.
|
|
1569
|
+
- **UNVERIFIED — skill dogfood.** The only agent CLI on
|
|
1570
|
+
this machine (codex) is not logged in, and no other agent
|
|
1571
|
+
CLI is installed, so no fresh agent session has yet been
|
|
1572
|
+
driven through the SKILL.md; no transcript is claimed.
|
|
1573
|
+
The skill's procedure is the exact command sequence the
|
|
1574
|
+
end-to-end smoke above executed by hand.
|
|
1575
|
+
|
|
1576
|
+
### Epic K3a (herdr pane hosting)
|
|
1577
|
+
|
|
1578
|
+
Agent seats named in the saved config's `[hosting]` table
|
|
1579
|
+
(`herdr = ["seat", ...]` in the user setup file) can now run
|
|
1580
|
+
inside herdr panes instead of headless subprocesses, so a run
|
|
1581
|
+
is watchable from herdr's UI. The `agent:` spec grammar is
|
|
1582
|
+
unchanged — hosting is a config hint, never part of a spec —
|
|
1583
|
+
and herdr is never a hard dependency.
|
|
1584
|
+
|
|
1585
|
+
- **Optional by construction** (new `herdrhost` module):
|
|
1586
|
+
detection probes, in order, `herdr` on PATH → version floor
|
|
1587
|
+
(`MIN_HERDR_VERSION` 0.9.0, set from the inspected 0.9.5
|
|
1588
|
+
tree) → feature-detect against `herdr api schema --json`
|
|
1589
|
+
(the used methods and the `pane.agent_status_changed` /
|
|
1590
|
+
`pane.exited` event names must be present) → a reachable
|
|
1591
|
+
session (`herdr api snapshot`). Any failure leaves the
|
|
1592
|
+
adapter inert: every seat runs headless exactly as before,
|
|
1593
|
+
and the run record carries exactly one `herdr_hosting` line
|
|
1594
|
+
saying why. With no hosting configured, records are
|
|
1595
|
+
byte-identical to pre-K3a records.
|
|
1596
|
+
- **CLI wrappers only** — no raw socket. The seat launches
|
|
1597
|
+
via `herdr pane run` (herdr's primitive for ordinary
|
|
1598
|
+
terminal processes; `agent start` is for interactive
|
|
1599
|
+
prompting of recognised agents). The review packet is
|
|
1600
|
+
never typed: the pane command redirects stdin from the
|
|
1601
|
+
prompt file, and `$CODEREVIEW_PROMPT` is set via
|
|
1602
|
+
`workspace create --env`. Output and the exit code land in
|
|
1603
|
+
files, and supervision reads process truth from them —
|
|
1604
|
+
the same capture files, pattern table, and terminal
|
|
1605
|
+
classifications as the headless runner, with identical
|
|
1606
|
+
error messages. A hosted seat's run leaves a `hosting`
|
|
1607
|
+
receipt in the record (transport, herdr version, pane,
|
|
1608
|
+
advisory observations), next to `seat_state`.
|
|
1609
|
+
- **Advisory pane state, failure discipline.** Herdr's
|
|
1610
|
+
`agent_status` is signal #4 (K2): it annotates the receipt
|
|
1611
|
+
and may raise an advisory `blocked:input` episode in the
|
|
1612
|
+
seat-state trace; it never decides an outcome, and when it
|
|
1613
|
+
disagrees with the files, the files win and the
|
|
1614
|
+
disagreement stays in the receipt. There is no `events`
|
|
1615
|
+
CLI in herdr, so the subscription is a supervised `pane
|
|
1616
|
+
get` watch; a break in it is treated as `events_lost` and
|
|
1617
|
+
reconciled via `session.snapshot` — same pane, same
|
|
1618
|
+
terminal, or the seat fails. Herdr dying mid-run, a pane
|
|
1619
|
+
exiting early, or a restarted session are recorded seat
|
|
1620
|
+
failures; a resumed session is never silently continued
|
|
1621
|
+
and no command is ever re-run inside an attempt.
|
|
1622
|
+
- **Environment discipline** (spec U7): we add only
|
|
1623
|
+
`CODEREVIEW_PROMPT` (a path) to the pane — plus `HOME` for
|
|
1624
|
+
a `home=private` seat. Operator API keys are never added;
|
|
1625
|
+
the pane session's own environment is the user's, which is
|
|
1626
|
+
the point of hosting a subscription CLI.
|
|
1627
|
+
- **UNVERIFIED — live smoke.** Real herdr is not installed
|
|
1628
|
+
on the build machine (and there is no Rust toolchain to
|
|
1629
|
+
build it), so the spec's live-smoke leg — a real herdr
|
|
1630
|
+
session hosting two seats through a real gate run — has
|
|
1631
|
+
NOT been run. The full lifecycle, hosted/headless
|
|
1632
|
+
equivalence, and the chaos cases (herdr dies mid-run,
|
|
1633
|
+
event break + reconcile, vanished pane, restarted
|
|
1634
|
+
session, contradictory pane state) are covered against a
|
|
1635
|
+
stub `herdr` executable implementing the used CLI surface
|
|
1636
|
+
(31 new tests; 30 fail on pre-K3a code, the one pass being
|
|
1637
|
+
the byte-identity guard for unchanged setups). The
|
|
1638
|
+
adapter's reading of herdr's response envelopes is taken
|
|
1639
|
+
from herdr's source and docs, not from a live session.
|
|
1640
|
+
|
|
1641
|
+
### Epic K2 (seat-state supervision)
|
|
1642
|
+
|
|
1643
|
+
Every seat in a run now carries a live state — pending →
|
|
1644
|
+
working → done / blocked:<class> / failed / timed-out, with
|
|
1645
|
+
blocked classes quota, auth, input, stalled — shown on the
|
|
1646
|
+
CLI's stderr as transitions happen and persisted per seat in
|
|
1647
|
+
the record (`seat_state`, plus `seat_state_primary` when a
|
|
1648
|
+
fallback attempt determined the outcome), alongside the
|
|
1649
|
+
existing error_detail machinery. A blocked seat degrades the
|
|
1650
|
+
panel exactly as a seat failure does today (fallback with the
|
|
1651
|
+
reason recorded, or degraded → INCONCLUSIVE); state itself
|
|
1652
|
+
never changes a verdict.
|
|
1653
|
+
|
|
1654
|
+
- **Detection is ours, over subprocess signals** (new
|
|
1655
|
+
`seatstate` module): exit codes, captured output, activity,
|
|
1656
|
+
timeout. The per-CLI failure-pattern table is probe-grounded
|
|
1657
|
+
(the K spec's U6 matrix): Codex auth patterns are taken from
|
|
1658
|
+
a live logged-out capture of codex-cli 0.164.0-alpha.9, its
|
|
1659
|
+
quota text from the vendor's own source; Claude's quota/auth
|
|
1660
|
+
text from the official error reference; Gemini's quota text
|
|
1661
|
+
from the official FAQ. Gemini auth text is UNVERIFIED in the
|
|
1662
|
+
docs, so gemini ships no auth pattern; the `input` class is
|
|
1663
|
+
wired but pattern-less for the same reason. Patterns scan
|
|
1664
|
+
stderr while a seat runs and, after a non-zero exit, stderr
|
|
1665
|
+
then stdout (Claude prints in-run failures as its result);
|
|
1666
|
+
a successful run's stdout is never scanned.
|
|
1667
|
+
- **Supervised agent wait** (`providers.command_agent`): the
|
|
1668
|
+
single communicate() wait is now a poll loop over the
|
|
1669
|
+
capture files — a verified blocked pattern followed by a
|
|
1670
|
+
grace period of total silence terminates a stuck seat early
|
|
1671
|
+
as blocked; a seat killed by its timeout while silent past
|
|
1672
|
+
the stall bound ends blocked:stalled, while still producing
|
|
1673
|
+
output it ends timed-out. Stall episodes are live/recorded
|
|
1674
|
+
liveness signals, never kills by themselves. All existing
|
|
1675
|
+
ProviderError messages are unchanged; errors now also carry
|
|
1676
|
+
`seat_facts` for the record.
|
|
1677
|
+
- **Discovery offers all three CLIs** (U6-verified forms):
|
|
1678
|
+
Codex (`codex exec -s read-only --skip-git-repo-check -`;
|
|
1679
|
+
form verified live + docs, end-to-end success not live-
|
|
1680
|
+
verified — no login on the build VM) and Gemini CLI
|
|
1681
|
+
(`gemini -p "…"`; docs-verified only), each entry carrying
|
|
1682
|
+
its provenance in comments.
|
|
1683
|
+
- **Not done, deliberately:** herdr pane-state annotation —
|
|
1684
|
+
seats are not herdr-hosted until K3a, so there is no pane
|
|
1685
|
+
state to read; when hosting lands it will be recorded as an
|
|
1686
|
+
advisory annotation only, never as the basis of a blocked
|
|
1687
|
+
classification (this track's classifications are ours alone,
|
|
1688
|
+
per the K2 spec).
|
|
1689
|
+
- Guard/test updates that ride along: the v14 subprocess
|
|
1690
|
+
guard accepts the supervised seat wait as a Popen pairing
|
|
1691
|
+
(it enforces the same timeout as a deadline + tree kill);
|
|
1692
|
+
one scripted-wizard test pins PATH so host-installed CLIs
|
|
1693
|
+
cannot add prompts to its script.
|
|
1694
|
+
|
|
1695
|
+
### 2026-11-05 (empty-content failure class)
|
|
1696
|
+
|
|
1697
|
+
- **Seat `ProviderError(kind="empty")` is now a content-class
|
|
1698
|
+
failure in the bench generation layer**
|
|
1699
|
+
(`bench/generation.py`). The DeepSeek soak (2026-11-05)
|
|
1700
|
+
reproduced the class on 5/24 bench calls against a healthy
|
|
1701
|
+
account — the provider answers with an empty content
|
|
1702
|
+
channel, its completion budget spent in the reasoning
|
|
1703
|
+
channel. Previously the generation layer classified it
|
|
1704
|
+
"unknown" (transport class) and latched the seat exhausted,
|
|
1705
|
+
losing the report as infrastructure noise. Now the seat
|
|
1706
|
+
records a `content_invalid` event (error class `empty`) and
|
|
1707
|
+
hands the failure to the product's own retry behaviour as
|
|
1708
|
+
the re-draw, with the pre-declared per-lens fallback seat
|
|
1709
|
+
next — no transport-budget burn, no latch. The raw
|
|
1710
|
+
empty-body transport return stays transport class; all
|
|
1711
|
+
other mappings unchanged. Equivalence-checked against the
|
|
1712
|
+
pre-change tree on public-split scripted scenarios
|
|
1713
|
+
(identical except for `kind="empty"` outcomes); 4 new
|
|
1714
|
+
parent-fail-verified tests. Detail:
|
|
1715
|
+
`bench/VOID-SOLUTION-FINDINGS.md` ("Empty-content class
|
|
1716
|
+
fix"). No change to policy B1, thresholds, metric
|
|
1717
|
+
definitions, gate semantics, or the confirmation scorer.
|
|
1718
|
+
|
|
1719
|
+
### 2026-11-05 (void solution: generation machinery, S1, S4)
|
|
1720
|
+
|
|
1721
|
+
Implements the machinery of the adopted post-void solution
|
|
1722
|
+
(`codereview-baseline-v2-void-solution-2026-11-05.md` §3) plus
|
|
1723
|
+
findings S1 and S4 from its §6. Findings status: **S1 fixed**
|
|
1724
|
+
(this entry), **S4 fixed** (this entry); **S2 and S3 are handled
|
|
1725
|
+
by the registration-addendum track, not here.** Nothing here
|
|
1726
|
+
changes policy B1, any threshold, any metric definition, or the
|
|
1727
|
+
confirmation scorer.
|
|
1728
|
+
|
|
1729
|
+
- **Generation/scoring separation for the bench**
|
|
1730
|
+
(`bench/generation.py`, driver
|
|
1731
|
+
`bench/scripts/run_generation_v2.py`): seat responses are
|
|
1732
|
+
generated per case x seat through a record/replay cache keyed
|
|
1733
|
+
by (seat, model, params hash, prompt hash). Transport errors
|
|
1734
|
+
are never cached; the first valid response (valid = the
|
|
1735
|
+
product's own parser accepts it) is cached and never
|
|
1736
|
+
re-drawn. Failure classes per §3.2: transport/infrastructure
|
|
1737
|
+
failures retry with backoff + jitter inside a per-job attempt
|
|
1738
|
+
cap and wall-clock budget and are not degraded when a valid
|
|
1739
|
+
response lands; content failures follow the product's own
|
|
1740
|
+
repair behaviour and then the pre-declared per-lens fallback
|
|
1741
|
+
seat (healthy when the fallback serves, `served_by` /
|
|
1742
|
+
`fallback_for` recorded); a seat unserved beyond budget marks
|
|
1743
|
+
the case `unscored-infra`. Generation is resumable — a run is
|
|
1744
|
+
complete when every case x configured seat has a valid
|
|
1745
|
+
response — and the frozen attempt file holds complete cases
|
|
1746
|
+
only. The committed confirmation scorer consumes the output
|
|
1747
|
+
unchanged (adapter: `generation.to_attempt_record`).
|
|
1748
|
+
Generation records carry per-case failure counts, and
|
|
1749
|
+
`bench/scripts/analyze_failure_features.py` implements the
|
|
1750
|
+
§3.2 post-hoc guard (failures/retries vs diff size, language,
|
|
1751
|
+
defect/clean).
|
|
1752
|
+
- **S4 — cost is metered, not declared:** the generation path
|
|
1753
|
+
records tokens per job and shadow-prices them from a list
|
|
1754
|
+
price table (labelled shadow/approximate in the code and the
|
|
1755
|
+
run manifest); attempt records from this path carry the
|
|
1756
|
+
metered `cost_usd` instead of the `cost_per_call_usd: 0`
|
|
1757
|
+
convention.
|
|
1758
|
+
- **S1 — lenses bind to seats explicitly (product fix):**
|
|
1759
|
+
`core.run_review(..., seat_lenses={seat: lens})` binds each
|
|
1760
|
+
seat to its lens in panel config. Excluding a seat leaves its
|
|
1761
|
+
lens unserved — served by the lens's configured fallback
|
|
1762
|
+
(recorded with `fallback_for` naming the absent seat) or the
|
|
1763
|
+
run is degraded with a "lens unserved" cause and cannot PASS
|
|
1764
|
+
— never a silent remap onto the surviving seats, which is
|
|
1765
|
+
what confirmation attempts 3–4 did. Bench configs accept
|
|
1766
|
+
`seat_lenses`; configs without one get the binding derived
|
|
1767
|
+
once at load from the provider order (the historical rule,
|
|
1768
|
+
recorded on the loaded config and in each report's
|
|
1769
|
+
`seat_lenses` section). Config identity: an explicit binding
|
|
1770
|
+
joins the config hash; a derived one does not (it is a pure
|
|
1771
|
+
function of the provider list already hashed, so pre-S1
|
|
1772
|
+
config hashes — and their stored records — are unchanged).
|
|
1773
|
+
- **Verification-round fixes on the generation driver** (the
|
|
1774
|
+
internal review blocked this branch on one major + four
|
|
1775
|
+
minors; all fixed, detail in
|
|
1776
|
+
`bench/VOID-SOLUTION-FINDINGS.md`): the registered-set guard
|
|
1777
|
+
now refuses every non-public split by name — holdout,
|
|
1778
|
+
expanded-clean, any future split — before any seat call,
|
|
1779
|
+
unless `CODEREVIEW_REGISTERED_RUN=1`; fallbacks are validated
|
|
1780
|
+
at driver entry (panel membership; under an explicit binding
|
|
1781
|
+
a fallback may not reassign a lens to a different seat) and
|
|
1782
|
+
the effective fallbacks join the run's identity in the
|
|
1783
|
+
generation manifest; budgets are stated and enforced as one
|
|
1784
|
+
model — a 600 s per-job wall-clock cap inside an 1,800 s
|
|
1785
|
+
per-case budget (per-seat case deadlines in
|
|
1786
|
+
`bench/generation.py`).
|
|
1787
|
+
|
|
1788
|
+
### 2026-11-05 (Epic K1: setup CLI discovery)
|
|
1789
|
+
|
|
1790
|
+
First-run setup now looks at the machine before asking for
|
|
1791
|
+
provider specs (`src/codereview/discovery.py` + the wizard's
|
|
1792
|
+
discovery step). It scans PATH for the agent CLIs the product
|
|
1793
|
+
already knows how to seat — Claude Code, Codex, Gemini CLI —
|
|
1794
|
+
reports each found binary with its resolved path and a cheap
|
|
1795
|
+
`--version` probe outcome (failures are printed as failures,
|
|
1796
|
+
never smoothed over), and offers the ones carrying a verified
|
|
1797
|
+
non-interactive invocation as panel seats: "they review using
|
|
1798
|
+
your existing CLI logins, no API key needed." Today exactly one
|
|
1799
|
+
invocation is verified in-tree — Claude Code's, the shipped
|
|
1800
|
+
example wrapper's `claude -p --tools "" ...` — so Codex and
|
|
1801
|
+
Gemini CLI are reported as found but not pre-filled, until a
|
|
1802
|
+
probe-verified template enters the table (spec U6). A confirmed
|
|
1803
|
+
pick saves an ordinary `agent:<name>:<command>` spec through
|
|
1804
|
+
the existing save path; declining everything leaves the flow
|
|
1805
|
+
and the saved config byte-identical to before. When discovery
|
|
1806
|
+
contributes seats, a family-diversity check warns in plain
|
|
1807
|
+
words if the confirmed panel covers a single vendor family
|
|
1808
|
+
and offers one API spec as the second family (never blocks
|
|
1809
|
+
saving). Non-interactive runs are untouched: no setup still
|
|
1810
|
+
stops with setup instructions. Tests:
|
|
1811
|
+
`tests/test_discovery_wizard.py` (22; fake PATHs/stub
|
|
1812
|
+
executables throughout, plus a hermetic end-to-end where a
|
|
1813
|
+
discovered stub seat serves a real review).
|
|
1814
|
+
|
|
1815
|
+
### 2026-11-05 (step-3 error persistence)
|
|
1816
|
+
|
|
1817
|
+
Implements step-3 recommendation 3 of the void forensics
|
|
1818
|
+
(`void-forensics-2026-11-05.md`): the bench record writer used
|
|
1819
|
+
to trim reviewer reports to provider/lens/status/attempts,
|
|
1820
|
+
dropping the seat error string — the voided confirmation
|
|
1821
|
+
attempts' exact failure classes were unrecoverable from their
|
|
1822
|
+
records. Additive only; no change to policy B1, thresholds,
|
|
1823
|
+
metric definitions, gate semantics, or the confirmation scorer.
|
|
1824
|
+
|
|
1825
|
+
- **Errored reports now persist their failure detail** on every
|
|
1826
|
+
record-writing path. Reviewer reports carry `duration_ms`
|
|
1827
|
+
(wall-clock of the call) and `prompt_chars` (prompt size in
|
|
1828
|
+
characters), and an errored report — or one whose primary
|
|
1829
|
+
failed before a fallback served — carries an `error_detail`
|
|
1830
|
+
block: the FULL error string, its `http_status` (parsed from
|
|
1831
|
+
the error by `core._http_status_of`; `null` when the error
|
|
1832
|
+
carries none — never invented), the failure classes, and the
|
|
1833
|
+
fallback's error/status when a fallback also failed.
|
|
1834
|
+
- **One shared persisted form**: `core.reviewer_report_entry`
|
|
1835
|
+
is now used by both the bench result writer
|
|
1836
|
+
(`src/codereview/bench.py`) and the generation attempt-record
|
|
1837
|
+
adapter (`bench/generation.py`), so no writer trims a report
|
|
1838
|
+
on its own. Generation per-case records fold each failure
|
|
1839
|
+
event's detail (error string, HTTP status, duration, prompt
|
|
1840
|
+
size) into an `error_detail` list, and generation events keep
|
|
1841
|
+
up to 2,000 chars of an error's text.
|
|
1842
|
+
- **Legacy records are unaffected**: no record hash covers
|
|
1843
|
+
reviewer reports, readers treat a missing `error_detail` as a
|
|
1844
|
+
legacy record, and the shipped v1 baseline results still load
|
|
1845
|
+
(pinned by test).
|
|
1846
|
+
|
|
1847
|
+
### 2026-11-05 (acceptance wiring, protocol v2)
|
|
1848
|
+
|
|
1849
|
+
- `acceptance.run_acceptance_v2`: the redesigned Tech Lead
|
|
1850
|
+
(techlead.adjudicate) can now be scored over the acceptance
|
|
1851
|
+
corpus through the SAME scorer and metric computation as
|
|
1852
|
+
the v1 runner (`_scorecard_metrics`, shared). V2 entries
|
|
1853
|
+
are mapped by `adapt_v2_entry` field-for-field; the
|
|
1854
|
+
two-field reasoning assessment the machinery stratum
|
|
1855
|
+
scores is the seat's own (the v2 verdict draw now returns
|
|
1856
|
+
it in the v1 vocabulary) — never derived by the adapter.
|
|
1857
|
+
Per-path results (deterministic probe path vs model path)
|
|
1858
|
+
are reported separately in the run record, never blended.
|
|
1859
|
+
M1 follows the scorecard's counting rules (by scored
|
|
1860
|
+
decision); the scorer's per-case false_dismissal flag is
|
|
1861
|
+
vestigial and unused. M4 is not run in v2 baseline mode.
|
|
1862
|
+
|
|
1863
|
+
### 2026-11-05
|
|
1864
|
+
|
|
1865
|
+
- Baseline v2 confirmation (step 7): VOID — no counting attempt
|
|
1866
|
+
completed. The pre-registered confirmation of policy B1 on the
|
|
1867
|
+
A5 holdout + expanded clean stratum was started six times on
|
|
1868
|
+
2026-11-05; every start was voided or aborted on
|
|
1869
|
+
provider-environment grounds (the DeepSeek seat errored in
|
|
1870
|
+
every degraded case; the NVIDIA free tier was quota-dead at
|
|
1871
|
+
three preflights; Gemini dead at every preflight, excluded as
|
|
1872
|
+
in the v1 baseline). No confirmation metric was measured and
|
|
1873
|
+
no pre-registered threshold is met or failed; B1 remains
|
|
1874
|
+
unconfirmed and the default gate profile stays `strict`.
|
|
1875
|
+
Attempts consumed under the registration: 0 (voids do not
|
|
1876
|
+
count); the sets remain unconsumed under the same
|
|
1877
|
+
pre-registration. Full accounting:
|
|
1878
|
+
bench/BASELINE-V2-2026-11-05.md (the standing quotable baseline
|
|
1879
|
+
remains bench/BASELINE-2026-11-04.md).
|
|
1880
|
+
- Gate profiles (step 6 of the adopted honest-negatives
|
|
1881
|
+
solution; candidate frozen by the step-4 replay sweep):
|
|
1882
|
+
policy B1 (per-lens corroboration) ships as the `balanced`
|
|
1883
|
+
gate profile. A blocking-severity finding whose merged
|
|
1884
|
+
sources are solely the operations lens — no
|
|
1885
|
+
correctness-lens or deterministic corroboration within the
|
|
1886
|
+
gate's merge identity — is demoted to a visible minor
|
|
1887
|
+
before archived/fixture/precedent/platform classification
|
|
1888
|
+
and before cross-family confirmation; the demotion and its
|
|
1889
|
+
reason are recorded on the finding (`profile_demotion`)
|
|
1890
|
+
and in the record's `gate_profile` section (effective
|
|
1891
|
+
profile, its source, every demotion). Selection:
|
|
1892
|
+
`--gate-profile` flag, else the policy file's new `[gate]
|
|
1893
|
+
profile` key, else the default. **The default remains
|
|
1894
|
+
`strict`** — today's behaviour exactly — under the freeze
|
|
1895
|
+
terms: B1 is confirmed once on the holdout in step 7
|
|
1896
|
+
(Baseline v2) before any default change. Findings now
|
|
1897
|
+
record per-source lens attribution (`source_lenses`,
|
|
1898
|
+
unioned by the merge; an unattributed source fails safe
|
|
1899
|
+
and is never demoted on a guess) and evidence-class
|
|
1900
|
+
metadata (E0 deterministic / E1 corroborated / E3
|
|
1901
|
+
single-seat unverified; E2 probe-confirmed is not assigned
|
|
1902
|
+
at gate time, since probes run in adjudication) — the
|
|
1903
|
+
classes change no verdict under `strict` and act only
|
|
1904
|
+
through B1 under `balanced`. **`quiet` is not implemented,
|
|
1905
|
+
by decision**: its blanket-corroboration form was rejected
|
|
1906
|
+
on the sweep evidence (it demoted three real single-seat
|
|
1907
|
+
majors, D6–D8), and naming it — CLI, policy file, or API —
|
|
1908
|
+
is a loud error, so the absence is a recorded decision,
|
|
1909
|
+
not an omission. Expected effect, quoted AS REPLAY
|
|
1910
|
+
(step-4 sweep, public split, adjudicated semantic
|
|
1911
|
+
grouping; live merge-identity grouping sits between strict
|
|
1912
|
+
and this row): blocking recall 29/34 enlarged (24/24
|
|
1913
|
+
labelled), merged blocking precision 1.000 excl. unknowns
|
|
1914
|
+
(0.967 incl.), clean-case BLOCKED 0.20 vs strict's 0.40.
|
|
1915
|
+
A claim-lint test (tests/test_claim_lint.py) now fails if
|
|
1916
|
+
the README, this CHANGELOG, or the baseline
|
|
1917
|
+
(bench/BASELINE-2026-11-04.md) quote the headline figures
|
|
1918
|
+
without the baseline's `required_caveat` adjacent, or
|
|
1919
|
+
present the superseded naive figure as a current claim.
|
|
1920
|
+
31 new tests (suite 802).
|
|
1921
|
+
- Tech Lead redesign, adjudication protocol v2 (branch
|
|
1922
|
+
`tl-redesign`; honest-negatives solution §3.3 + §3.5,
|
|
1923
|
+
owner decision 2026-11-05 to proceed despite the §3.7
|
|
1924
|
+
tally): new module `src/codereview/techlead.py`, protocol
|
|
1925
|
+
v1 (`core.shadow_adjudicate`) untouched and still the
|
|
1926
|
+
wired shadow path. The v2 seat returns verdict +
|
|
1927
|
+
calibrated confidence + deciding quote only — "escalate"
|
|
1928
|
+
and "needs_probe" are invalid outputs; the harness
|
|
1929
|
+
escalates on exactly six named rules (protected class,
|
|
1930
|
+
independence, draw disagreement, evidence ineligibility,
|
|
1931
|
+
major dismissal without probe support, low confidence),
|
|
1932
|
+
each recorded on the entry. Settling probes decide on a
|
|
1933
|
+
deterministic path where the seat only grades probe
|
|
1934
|
+
adequacy; the two paths are reported separately, never
|
|
1935
|
+
blended. Packets are canonicalised (vendor/family
|
|
1936
|
+
scrubbed, content-keyed position order), so vendor-strip
|
|
1937
|
+
and family-swap invariance holds BY CONSTRUCTION and is
|
|
1938
|
+
labelled as such. Double draw is canonical + swapped;
|
|
1939
|
+
majors always draw twice. Judging frame is AS RAISED —
|
|
1940
|
+
a fix shipped in response to a finding supports it
|
|
1941
|
+
(FIX_SHIPPED settles to sustain_finding; the L06
|
|
1942
|
+
inversion is structural now). Dismissal-quote eligibility
|
|
1943
|
+
consumes `core._dismissal_quote_check` from the harness
|
|
1944
|
+
fixes (call-time resolution, grounding-only fallback
|
|
1945
|
+
labelled on the record). Every entry carries its §3.5
|
|
1946
|
+
authority stage (1/2/3/never); `ENABLED_STAGES` is empty
|
|
1947
|
+
— hooks built, nothing flipped, no acceptance or
|
|
1948
|
+
evaluation run made. 47 new tests (suite 759).
|
|
1949
|
+
- Issue 2 (benchmark precision) CLOSED by the measurement path
|
|
1950
|
+
(owner act): blind probe-first adjudication of all 91 unmatched
|
|
1951
|
+
findings — 64 duplicates, 10 real unlabelled defects (added to
|
|
1952
|
+
ground truth), 7 true false positives. Corrected semantic
|
|
1953
|
+
precision 0.870 finding-level / 0.756 merged (blocking 0.956 /
|
|
1954
|
+
0.906); the baseline's required_caveat carries the corrected
|
|
1955
|
+
figures and the naive figure is superseded. Behaviour work
|
|
1956
|
+
(policy profiles, corpus expansion, Baseline v2) continues.
|
|
1957
|
+
- Harness fixes from the honest-negatives step-2 forensics
|
|
1958
|
+
Part A (branch `harness-fixes`; no Tech Lead redesign, no
|
|
1959
|
+
gate change, no acceptance re-run): (1) the
|
|
1960
|
+
dismissal-evidence rule is enforced in shadow draws (L06)
|
|
1961
|
+
— a dismissal's deciding quote must be distinct from the
|
|
1962
|
+
refutation's quote and pass a deterministic relevance
|
|
1963
|
+
screen against the claim, or the dismissal is
|
|
1964
|
+
evidence-ineligible and escalates; the acceptance scorer
|
|
1965
|
+
applies the same full check. (2) Cited-line discipline
|
|
1966
|
+
for presented regions (P06) — `locate_presented_region`
|
|
1967
|
+
prefers the anchor window containing the finding's cited
|
|
1968
|
+
line, centres on the cited line when no anchor window
|
|
1969
|
+
contains it, and marks `region_unverified` when the line
|
|
1970
|
+
cannot be placed; the corpus builder delegates to it
|
|
1971
|
+
(P06's inputs now locate at 938–954, the actual defect,
|
|
1972
|
+
instead of the sibling at 609–625), and
|
|
1973
|
+
`acceptance.case_inputs` flags inputs whose stored region
|
|
1974
|
+
is verified by neither the cited line nor the finding's
|
|
1975
|
+
grounded quote (9 of 48 shipped cases, P06 among them),
|
|
1976
|
+
carried loudly on the shadow entry.
|
|
1977
|
+
|
|
1978
|
+
|
|
1979
|
+
## 0.12.1 — 2026-11-05 — acceptance fixes
|
|
1980
|
+
|
|
1981
|
+
Patch release from the clean-room acceptance test of the built
|
|
1982
|
+
0.12.0 package (verdict: READY-WITH-DEFECTS). The planted defect
|
|
1983
|
+
was caught precisely and the first-run, policy, evidence and
|
|
1984
|
+
record-verify surfaces behaved as designed, but a clean, correct
|
|
1985
|
+
change could be BLOCKED by confident false findings. 0.12.0 was
|
|
1986
|
+
never published to PyPI (tag only); the corrected build ships
|
|
1987
|
+
as 0.12.1. Suite 870.
|
|
1988
|
+
|
|
1989
|
+
- **D1 — file context on the standard diff path:** reviewers
|
|
1990
|
+
previously received only the plain unified diff (3 lines of
|
|
1991
|
+
context), so definitions outside the hunk were invisible and
|
|
1992
|
+
correct code drew false "undefined symbol" majors from two
|
|
1993
|
+
model families at 0.98 confidence. The diff path now attaches
|
|
1994
|
+
the post-change contents of changed files, with per-file and
|
|
1995
|
+
total caps, in-text truncation markers, and the context budget
|
|
1996
|
+
allocated to the most-changed files first. `run_review` gains
|
|
1997
|
+
`file_context=None` — bench and registered-run prompt shapes
|
|
1998
|
+
stay byte-identical by default; the acceptance clean case now
|
|
1999
|
+
reaches PASS while the planted defect still BLOCKs.
|
|
2000
|
+
- **D2 — stored error type/status:** seat error strings stored
|
|
2001
|
+
in run records now preserve the HTTP status and exception
|
|
2002
|
+
type instead of truncating mid-token, so a rate limit (429)
|
|
2003
|
+
is distinguishable from a crash in the record.
|
|
2004
|
+
- **D3 — `--help` subcommand list:** top-level `codereview
|
|
2005
|
+
--help` now lists the subcommands (setup, policy,
|
|
2006
|
+
record-verify, doctor, …) instead of leaving them
|
|
2007
|
+
discoverable only through the README.
|
|
2008
|
+
- **D4 — same-line finding merge:** identical findings raised
|
|
2009
|
+
by two seats on the same line are merged into one finding
|
|
2010
|
+
listing all raising seats, and the gate counts it once.
|
|
2011
|
+
|
|
2012
|
+
## 0.12.0 — 2026-11-04 — the finish release
|
|
2013
|
+
|
|
2014
|
+
The clean-gate hold is LIFTED by the owner on an eligible Form B chain
|
|
2015
|
+
(complete-coverage round 1 over 3013656..2fbf94f + 2 verification
|
|
2016
|
+
rounds; record-verify --chain 12/12, exit 0). Suite 691.
|
|
2017
|
+
|
|
2018
|
+
- Dispute machinery: computed inventory (codereview disputes),
|
|
2019
|
+
append-only disposition store (116 owner-ratified entries),
|
|
2020
|
+
authority-affecting tagger, quote-anchored locate, release-flow
|
|
2021
|
+
freshness check in record-verify --chain.
|
|
2022
|
+
- Epic A benchmark + first quotable baseline: recall 24/24 on the
|
|
2023
|
+
public split (Wilson 95% lower bound 0.86), escaped-defect rate
|
|
2024
|
+
0.000; deterministic layer alone recall 0.458. Precision: the
|
|
2025
|
+
naive one-to-one figure as first published (superseded the
|
|
2026
|
+
same day by the adjudicated recount — never quoted; the
|
|
2027
|
+
corrected figures are in the measurement-repair addendum,
|
|
2028
|
+
bench/BASELINE-2026-11-04.md §6; the naive count's other
|
|
2029
|
+
rows: blocking-tier 0.267, merged-finding 0.727 with 82
|
|
2030
|
+
duplicate matches removed); clean-case BLOCKED rate 0.40 (2/5) and
|
|
2031
|
+
clean-case any-finding rate 0.60 (3/5) are the stated caveats —
|
|
2032
|
+
corrected figures in the measurement-repair addendum,
|
|
2033
|
+
bench/BASELINE-2026-11-04.md §6 (adopted solution §2.2).
|
|
2034
|
+
- Epic B policy file, Epic C evidence runner, Epic D1 SARIF export.
|
|
2035
|
+
- Probe harness (sandboxed, rebut-only) + shadow Tech Lead (full
|
|
2036
|
+
adjudication schema, A/B blinding, double-draw). Acceptance run
|
|
2037
|
+
EXECUTED: unlock NOT earned (M1/M2/M3/M4/M6 fail; M5 pass) — the
|
|
2038
|
+
Tech Lead remains shadow-only.
|
|
2039
|
+
- Verify mode repaired: archived-aware sectioned prompts and
|
|
2040
|
+
batched (never truncated) prior findings.
|
|
2041
|
+
- Renderer gate line states the degraded cause; duplicate-seat
|
|
2042
|
+
record fidelity; prompt-slice truncation markers inside the cap.
|
|
2043
|
+
|
|
2044
|
+
### process tooling (branch `process-tooling`)
|
|
2045
|
+
|
|
2046
|
+
Implements §5 of the adopted open-refutations solution
|
|
2047
|
+
(2026-11-04): the computed dispute inventory, the append-only
|
|
2048
|
+
disposition store, the release-flow freshness check, the
|
|
2049
|
+
deterministic authority-affecting tagger, and quote anchoring.
|
|
2050
|
+
|
|
2051
|
+
- **`codereview disputes`** scans all `review-*/` records (plus
|
|
2052
|
+
evidence sidecars) and prints the open-dispute inventory —
|
|
2053
|
+
findings with status `disputed` or `refutation_weak` and no
|
|
2054
|
+
disposition in the store — grouped by record with lens,
|
|
2055
|
+
raiser/refuter families, and the authority-affecting tag;
|
|
2056
|
+
`--json` for tooling. Run summaries now carry `open_disputes`
|
|
2057
|
+
(count + fingerprint list).
|
|
2058
|
+
- **Disposition store** `dispositions.jsonl` (repo root,
|
|
2059
|
+
append-only): entries carry the fingerprint, type
|
|
2060
|
+
(`sustain_refutation` / `sustain_finding` / `split` / `fixed`
|
|
2061
|
+
(requires the fixing commit) / `retired_archived_subject` /
|
|
2062
|
+
`retired_out_of_scope` / `superseded` / `downgraded`), decider
|
|
2063
|
+
(`owner` / `tech_lead` / `probe`), date, the reasoning-quality
|
|
2064
|
+
fields `reason_check` / `reason_corrected` / `basis_evidence`,
|
|
2065
|
+
and a `ratified` / `override` marker. `codereview disputes
|
|
2066
|
+
--record <entry.json>` appends with validation: a fingerprint
|
|
2067
|
+
is dispositioned once, unless the new entry is an override
|
|
2068
|
+
naming the entry it supersedes. `record-verify` consults the
|
|
2069
|
+
store automatically when present (a disputed finding with an
|
|
2070
|
+
entry counts as dispositioned; consumed entries are named);
|
|
2071
|
+
with no store, verification is unchanged.
|
|
2072
|
+
- **Freshness check:** `codereview disputes --check --span
|
|
2073
|
+
A..B` exits non-zero listing any dispute in records touching
|
|
2074
|
+
the span that is neither dispositioned nor listed in a
|
|
2075
|
+
caller-supplied `--inventory` file. Documented in the README
|
|
2076
|
+
as the release-flow gate; intentionally not wired into any
|
|
2077
|
+
automated script.
|
|
2078
|
+
- **Authority-affecting tagger** (deterministic, §5.5): the
|
|
2079
|
+
former "protected class", defined as findings that could
|
|
2080
|
+
change who decides, whether independence holds, whether a
|
|
2081
|
+
precedent/decision is valid, whether eligibility passes, or
|
|
2082
|
+
what leaves the machine. Report rendering and prompt display
|
|
2083
|
+
truncation are explicitly excluded; a manual override list
|
|
2084
|
+
wins over the rules.
|
|
2085
|
+
- **Quote anchoring** (§5.6): findings gain `line_basis`
|
|
2086
|
+
(`blob` / `diff`) when the cited line and quote determine one
|
|
2087
|
+
numbering; deterministic findings are `blob`. `codereview
|
|
2088
|
+
locate <finding-id> --record <path>` resolves a finding's
|
|
2089
|
+
validated quote (sidecar full text when truncated) to its
|
|
2090
|
+
blob position(s) at the reviewed commit; legacy findings
|
|
2091
|
+
without a basis still locate by quote.
|
|
2092
|
+
|
|
2093
|
+
### Epic B: project policy file
|
|
2094
|
+
Built on the `epic-b-policy` branch off v0.11.1; not yet merged or released. Implements Epic B of the adopted roadmap (`codereview-revised-plan-and-engineering-roadmap-2026-11-04.md` §11). The announcement hold is untouched.
|
|
2095
|
+
- **B1 — `.codereview.toml` schema (`src/codereview/policy.py`, new)**: the repository's own review contract, parsed with stdlib `tomllib` only — `[panel]` (min_seats, min_families, required_families, require_lenses), `[evidence]` (require_tests, require_tests_pass), repeatable `[[paths]]` rules (match, require_full_panel, require_tests, require_tests_pass, min_seats, min_families, enforce = block|advisory), and `[waivers]` (allow, require_reason, max_age_days). Every key is optional and the defaults are the documented no-requirement values, filled into an effective-policy dict that also records the file's path and sha256. Validation is strict (unknown tables/keys, wrong types — a TOML bool is not an int — negative counts, unknown lenses, a `[[paths]]` entry without `match`) and every error is line-referenced: TOML syntax errors carry tomllib's own line, semantic errors locate the offending key in its table. This is the PROJECT's policy, a different thing from the per-user setup config (`userconfig.py`), which only stores a person's provider specs.
|
|
2096
|
+
- **B2 — the gate reads policy**: `core.run_review(..., policy=..., waivers=...)` evaluates the loaded policy against the round's facts (changed files, reviewer reports, test results, recorded waivers) via the pure `policy.evaluate_policy` (path semantics are the engine's own `path_matches`, passed in — one implementation). The record gains a `policy` section listing every rule that applied (with satisfied/violated status) and every violation; violations join the gate reasons, and a blocking violation turns a PASS into BLOCKED (an INCONCLUSIVE round stays inconclusive, violations on the record). Advisory path-rule violations are recorded, never gate-changing. `render_markdown` shows the policy line and its violations. The CLI review command discovers `.codereview.toml` at the repo root or takes `--policy PATH`; an invalid file stops the run. Scope note: verification rounds (`--verify-previous`), `pr`, and `loop` do not take a policy yet.
|
|
2097
|
+
- **B3 — `codereview policy check`**: validates the discovered (or `--policy`-named) file and prints the effective policy with defaults filled in; with no file it prints the defaults and says so (a valid state, exit 0); an invalid file exits 2 with the line-referenced error.
|
|
2098
|
+
- Suite: 449 tests (49 new in `tests/test_policy.py`: loader validation + line references, per-rule evaluation incl. waiver ageing, gate integration with fake panels, and the CLI surface). Full suite green.
|
|
2099
|
+
|
|
2100
|
+
### Epic C: evidence runner (branch `epic-c-evidence`)
|
|
2101
|
+
|
|
2102
|
+
The evidence runner (roadmap Epic C, Week 2): the deterministic counterpart to the reviewer panel. Reviewers judge a diff but cannot run anything; the evidence runner executes the project's own configured commands and security adapters against the change, stores a content-hashed record in the run JSON, and makes required evidence a first-class gate input. The announcement hold stays ON; nothing here is a closing attempt.
|
|
2103
|
+
|
|
2104
|
+
- **C1 — `codereview.evidence` (new module) + `collect()`**: configured commands (tests, lint, SAST) execute in a sandboxed working copy of the tree; every item records status (`passed`/`failed`/`timeout`/`unavailable`/`missing`/`error`), exit code, duration, the sha256 of its full output, and a redacted output tail. Timeouts are enforced by killing the child's whole process group. Attached to a review via `--evidence-config`, the record is stored verbatim in the report under `evidence`.
|
|
2105
|
+
- **C2 — adapters**: `gitleaks`, `semgrep`, and `dependency-audit` (first available of osv-scanner / pip-audit / npm-with-package.json), resolved against the caller's PATH and run through the same sandbox executor. A missing tool is recorded `unavailable` with the reason — never a crash. A config `require` list records policy-demanded-but-unconfigured evidence as `missing`: the mechanism by which a policy requires a tool's presence. The config dict shape deliberately mirrors the Epic B policy file's `[evidence]` table (B1), so a `tomllib`-parsed policy section can be handed to `collect()` unchanged when Epic B lands.
|
|
2106
|
+
- **C3 — gate rule**: required evidence failed / timed out / errored / missing ⇒ **BLOCKED**; required evidence unavailable ⇒ **INCONCLUSIVE** (the machine could not produce the evidence — not the builder's failure). Implemented as the pure `evidence.gate_effect()` over the item list and applied in `run_review` after every other gate adjustment; it can only move a gate toward blocked/inconclusive, never clear anything. Optional evidence is recorded and never moves the gate.
|
|
2107
|
+
- **C4 — builder-claim verification**: `parse_claims` extracts the claims a builder work report actually commits to (backtick-quoted command + outcome on the same line, or a labelled `Tests:`/`Ran:`/`Command:` line with the outcome in its tail; outcome words inside the command itself never count, and commands without a claimed outcome are ignored, never guessed at). `verify_claims` re-runs each claim in the sandbox and compares exit outcomes: matching claims are `confirmed`; contradicted claims are **downgraded** in the record with claimed vs actual outcomes stated. Standalone surface: `codereview evidence --claims report.md`.
|
|
2108
|
+
- **C5 — sandbox safety, with the threat model documented in the module and the README**: commands run in a throwaway copy of the tree (`.git` excluded, symlinks never dereferenced during the copy, copy deleted after the run) — never in the reviewed checkout; the child environment is built from an allow-list (no provider API keys, tokens, or SSH agent; HOME/TMPDIR inside the sandbox); no network by default (proxy variables stripped; a Linux user+network namespace via `unshare -rn` where available — the record states `namespace` vs `env-only` honestly, and an item must opt in with `allow_network: true` to run unrestricted); POSIX rlimits (CPU time, per-file size) recorded in the record's `sandbox` block. Output tails pass through the review path's own `redact_secrets`, so a scanner that prints the secret it found cannot smuggle it into the record.
|
|
2109
|
+
- Suite: 435 tests (35 new in `tests/test_evidence.py` covering every ticket: record/hash/duration, timeout enforcement, copy isolation, env scrubbing with an operator marker that must not cross, honest network-mode recording, adapter present/absent/failing via fake tools on a test PATH, policy `require` → `missing`, every gate state as a pure function and end-to-end through `run_review`, claim parsing forms and downgrade-on-contradiction, claims verified inside the sandbox).
|
|
2110
|
+
- Example config: `examples/evidence.json`.
|
|
2111
|
+
|
|
2112
|
+
## 0.11.1 — 2026-11-04
|
|
2113
|
+
The v0.11.0 fix step: dispositions the standing items of the v0.11.0 internal review (record `review-v110/codereview-20261105-001944.json`, BLOCKED as received) — one deterministic blocker, one major, two minors, each cited by finding ID below. Nothing here adjudicates any disputed or refutation-weak finding; the owner's protected-class queue (nine items) is untouched, and the announcement hold stays ON.
|
|
2114
|
+
- **Blocker `44a6ae2d215a` — closed durably; the deterministic rule is NOT weakened.** The hardcoded-secret rule fired for the third time on a test fixture's fake key value — this time the `DEEPSEEK_API_KEY` fixture at `tests/test_user_setup.py:97`. The in-branch rename (`98ad1d9`) had removed the key-prefix shape but not the assignment shape the rule reads, which is why it re-fired; the correction to the v0.11.0 disposition bullet recorded in `0ba2c17` stands, is not rewritten, and this release is the fix it pointed at. Instead of renaming one string again, the whole fixture surface was audited and the class closed: **every string neutralized** (the full `tests/` tree scanned against the three deterministic secret rules plus bare key shapes) — (1) `tests/test_user_setup.py`: the fixture value `sentinel-value-never-stored` → `placeholder-deepseek-key-never-stored`, carrying the scan's own placeholder exemption token so the rule can never read it as a secret (the test's assertion — the value never reaches the saved config — is unchanged); (2) `tests/test_core.py`: the scanner/redaction fixture value is assembled at runtime from fragments (`FAKE_KEY_VALUE`, with a short placeholder token inside the `BUGGY` patch replaced at build time) — the runtime value is byte-identical, so the scan and redaction assertions are unchanged, but no literal line matches; (3) `tests/test_external_review_v070.py`: the agent-env fixture value likewise assembled from fragments (`FAKE_OPENAI_VALUE`), runtime value identical. **Audited and left as-is:** `ghp-test-SECRET` (`tests/test_external_review_v070.py`) — a hyphenated test marker that matches no secret pattern (GitHub's token shape is `ghp_` + alphanumerics). **Regression guard:** new `tests/test_fixture_secret_hygiene.py` runs the product's own deterministic scan (with its placeholder exemption) over every line under `tests/` presented as added diff lines — exactly what a review of a test-suite change sees — plus a bare key-shape scan; a fourth firing now fails the suite instead of surfacing in a review. Negative control verified: all three historical fixture shapes trip the guard.
|
|
2115
|
+
- **Major `3bbe648d8bfc` — doctor probe calls are wait-bounded.** `run_doctor` called each seat synchronously with no bound; a seat whose callable hung (a transport with no ceiling of its own) blocked the preflight forever. Probes now run through `_probe_call_bounded` — the same daemon-thread wait bound the review path gives reviewer / Tech Lead calls (`_review_job_bounded`) — with the same default semantics: `timeout=None` resolves to `TECH_LEAD_CALL_TIMEOUT`, and a non-positive timeout is a caller error (ValueError before any seat is probed). `codereview doctor --timeout` now also sets the probe bound (it already forwarded to agent-seat construction). A timeout surfaces as a failed seat (`error_class: transient`, "timed out" error), never a hang. Tests: `DoctorTimeoutTests` — hanging seat bounded and reported, fast seat unaffected, default-ceiling semantics, non-positive rejection, `_doctor_main` wiring.
|
|
2116
|
+
- **Minor `a509b806fda6` — the wizard's provider loop has a retry bound.** `MAX_PROVIDER_ATTEMPTS = 3` invalid or empty provider entries; exhausting the bound aborts through the same EOF channel as an EOF at any prompt (message printed, nothing saved, clean exit). Tests cover exhaustion (aborts, no config written) and recovery within the bound (bad entry, empty entry, then a valid one proceeds).
|
|
2117
|
+
- **Minor `84fc41a76d3f` — duplicate-role coverage for the lens roles.** `parse_role_assignments` already rejected a role assigned twice, but the only duplicate test used a non-lens role. Added explicit coverage for the dangerous case: a duplicated lens role (`correctness`, `operations`) is rejected at parse, and through `_apply_role_assignments` the rejection lands before any seat is built or merged — the panel is left untouched and no lens pin is recorded.
|
|
2118
|
+
- Suite: 400 tests (10 new: 1 fixture-hygiene guard, 5 doctor-timeout, 2 wizard-bound, 2 duplicate-lens-role).
|
|
2119
|
+
- **Internal review of this diff (verdict as received — no PASS is claimed)**: **INCONCLUSIVE**, not degraded, coverage complete, the record's own test run green (400 tests); **zero standing findings and zero deterministic findings** — blocker `44a6ae2d215a` did not fire on this diff, the outcome the hygiene guard exists to make permanent. Gemini quota-failed on the batch (not retried); the OpenAI fallback served correctness, NVIDIA operations. One **refutation-weak major** is recorded exactly as received: `a1982210ce99` (OpenAI-raised, NVIDIA-refuted) claims the `test_user_setup.py` fixture "still sets the old redacted value while the test asserts against a different string." Builder evidence, recorded without adjudicating: in the committed tree the assignment and the `assertNotIn` carry the identical placeholder string and the wizard test passes against it; the panel could not see that, because the product's own redaction rewrites any `KEY="…"` value to `[REDACTED]` before a diff leaves the machine — both the claim and the refutation reasoned from the redacted text, and the refutation's quote is the redacted line itself (hence weak adequacy). The finding therefore turns on what the evidence pipeline shows the panel, so it is flagged a **protected-class candidate** (evidence/redaction machinery) and joins the owner's ruling queue (now ten) exactly as recorded; no independent Tech Lead served (two families live), so the human gate decides. `record-verify` on the record passes every recomputation (patch hash, range, head currency, 6 evidence refs against their sidecars, lens coverage, two families) and fails only `gate_pass`, because the gate is INCONCLUSIVE. Record: `review-v111/codereview-20261105-003107.json`.
|
|
2120
|
+
|
|
2121
|
+
## 0.11.0 — 2026-11-04
|
|
2122
|
+
The user-configuration release: the `role-model-choice` branch (built in parallel in an isolated worktree) merges onto v0.10.0 — per-role model choice, a provider registry covering all major LLM providers, and first-run setup with NO shipped default panel (owner directives 2026-11-04). This release also mints the owner's 2026-11-04 sustained-refutation rulings into the precedent store. No closing attempt ships in this release; the Tech Lead gains no binding authority here — the probe harness and shadow-mode upgrade remain the next track. The announcement hold stays ON.
|
|
2123
|
+
- **Provider registry (`src/codereview/registry.py`, new)**: the single source of truth for provider presets — OpenAI, Anthropic, Google Gemini, DeepSeek, MiniMax, Moonshot (Kimi), NVIDIA NIM, xAI, Mistral, OpenRouter, a generic `openai-compatible:<base-url>` escape hatch, and agent seats. Each preset carries its base URL, key env var (bring-your-own-key; the product never supplies or stores a key), suggested model, **family tag** (feeding the Tech Lead independence check and the record), and request quirks (e.g. models that reject a sampling temperature get none sent — the gpt-5 lesson, encoded per preset). The seat builders in `providers.py` read family/kind from this registry, so a seat's family is a build-side fact, never a self-report.
|
|
2124
|
+
- **Per-role model choice**: `--role <role>=<provider-spec>` (roles: correctness, operations, verifier, tech-lead, solver) assigns the user's preferred model to each role; lens roles pin their seat in `run_review` (`lens_seats`), and the Tech Lead designation flows through the step4 seam. The record carries `seat_roles` and `role_models` (seat, provider, model, family, key env per role) — any verdict shows exactly which model served which role. With no assignment, the panel behaves exactly as before this release.
|
|
2125
|
+
- **First-run setup — no default panel (owner amendment, mid-build)**: `codereview setup` (new `userconfig.py` + `setup_wizard.py`) is an OpenClaw-style first-open flow: pick providers, name the key env vars (the wizard never asks for, echoes, or stores a key value — the saved config references env var names only), assign a model per role, save as YOUR defaults (`$CODEREVIEW_CONFIG`, else `$XDG_CONFIG_HOME/codereview/config.toml`). Provider resolution order is config-first: explicit `--provider` flags → `$CODEREVIEW_PROVIDERS` → the saved setup → (interactive TTY only) the wizard → stop. An unconfigured non-interactive run exits 2 with setup instructions; it never silently reviews with a built-in panel. Our internal Gemini/NVIDIA/OpenAI panel exists only as our own saved setup in the internal runner, never as a product default.
|
|
2126
|
+
- **Hand-merged integration seams (merge `96bfe03`)**: `core.run_review` keeps BOTH the step4 adjudication keywords (`tech_lead`, `probe_fn`, `tech_lead_timeout`) and `lens_seats`; `cli.py` composes `precedents` + `tech_lead_timeout` + the role `review_kwargs` at the review call, with the explicit `--tech-lead` flag taking precedence over a `--role`/saved-setup designation of the same role (command line beats config, as `--role` already did). The branch's one adapted test (`tests/test_v090_fixes.py`) carries over. Merged suite: 390 tests, green.
|
|
2127
|
+
- **Dispositions of the providers branch's own review residue — recorded honestly, nothing adjudicated by the builder**: the branch's two internal reviews (records in the branch worktree, `review-providers-out/`, `review-providers-out2/`) left the following. (1) Deterministic **blocker** `44a6ae2d215a` ("possible hardcoded secret in added code") fired on an `sk_live_…` fixture string in the wizard tests; it was **fixed in-branch** (`98ad1d9`, the sentinel renamed) and the merged tree is verified free of the pattern — the finding stands as recorded, a true positive against the fixture text at review time. (2) Refutation-weak major `19a537eb386b` (a role assignment could record the wrong seat, then crash on duplicate provider names) was **hardened in-branch by change**: `_apply_role_assignments` refuses loudly when a role seat's name is already owned by a seat of the other kind with proven tags, and build-side family tags remove the ambiguity the claim cited — addressed by change; the finding stands as recorded. (3) Refutation-weak major `931ea3882b76` (aborting the interactive first-run wizard crashed the review command) was **partially fixed in-branch** (`98ad1d9`: EOF/Ctrl-C at the wizard is caught at the `main()` call site and exits cleanly at 2); any remainder is recorded as-is. (4) Minor `353eaa52d669` (the wizard would re-trigger on every TTY first-open without checking for an existing setup) is answered by the config-first resolution above: the wizard branch is reached only when flags, env, AND the saved setup all yield no providers — the rebuttal the branch record carries, cited here, not adjudicated. (5) Two **disputed majors stand exactly as recorded**: `3d9645200bc5` (family tag not set for seats built from custom/openai-compatible specs without a registry entry) and `e1e22d48db4d` (`build_providers` does not apply the model override from a `preset:model` spec). Both touch the family/seat machinery, so both are flagged **protected-class candidates** and join the owner's ruling queue behind the three items from the v0.10.0 review (`4ce0212286e0`, `6b29fe953939`, `bf4c451e8751`) — the queue now holds five.
|
|
2128
|
+
- **Precedent store: the owner's 2026-11-04 rulings minted (store 5 → 19 entries)**: the 12 review-v15 sustained refutations (rulings addendum, v15 closing-run report) and the 4 step4 protected-class sustained refutations (refutation-process report) were converted with the v0.10.0 function-level tooling at HEAD (`upsert_precedent`), each entry citing its ruling source. 14 entries minted: 13 function-keyed to exactly the function each ruling concerns (`run_bench`, `percentile`, `main`, `_doctor_main`, `path_matches`, `run_verification`, `_parse_agent_spec`, `build_providers` ×2, `apply_precedents`, `_precedent_candidate`, `_select_tech_lead`, `_seat_meta`) and 1 file-keyed (`2242edbba9c9` — a module-scope imports claim with no enclosing function; the import block and `_kill_tree` were diff-verified unchanged since the ruling). Live check: re-raised claims classify `standing` at function and file scope; the precedent suites pass. **Two exceptions, not forced**: `3c39c4bdc15a` and `2fa2b04e08d5` cite CHANGELOG.md lines whose disposition text has since moved in a living changelog (the disposition blob no longer exists at the cited lines) and function keying does not apply to Markdown — the tooling cannot express them faithfully, so no entry was minted; the rulings themselves stand in the v15 report.
|
|
2129
|
+
- **Internal review of this diff (verdict as received — no PASS is claimed)**: **BLOCKED**, not degraded, coverage complete, the record's own test run green (390 tests); 2 findings withdrawn in the re-ground pass. Gemini quota-failed on every batch (not retried); the OpenAI fallback served correctness, NVIDIA operations. Standing findings: one deterministic **blocker**, `44a6ae2d215a` again — and a correction to the disposition bullet above is owed: the in-branch rename (`98ad1d9`) fixed the wizard-test sentinel, but the secret rule fired in this review on a different fixture string in the same file, the fake `DEEPSEEK_API_KEY` value at `tests/test_user_setup.py`:97; the merged tree is free of the `sk_live_`/`sk_test` patterns (verified), not of every fixture string the rule reads as a secret. The finding stands as received — a true positive against fixture text, the class the fixture-manifest mechanism exists to classify. One **major** stands (`3bbe648d8bfc`: `_doctor_main` missing timeout guard on provider calls, on the merged `cli.py`) and two minors (`a509b806fda6` wizard provider loop has no retry bound; `84fc41a76d3f` a missing duplicate-role parse test) — all recorded as received for the next fix step, none dispositioned here. Two majors are **disputed** (`f56cd4a0b6c9`, `_apply_role_assignments` seat re-assignment dedup; `e7fc93e0bb77`, `parse_role_assignments` empty-spec handling — both NVIDIA-raised) and two **refutation-weak** (`78ddf4d5ea27`, role assignment binding with duplicate preset seats; `9a4415ddafa6`, malformed roles value raising `AttributeError` — both OpenAI-raised): with only two families live (Gemini down) no independent Tech Lead could serve, so all four route to the human gate exactly as the design prescribes; all four touch the role/seat/config machinery, so all four are flagged **protected-class candidates** and join the owner's ruling queue (now nine: `4ce0212286e0`, `6b29fe953939`, `bf4c451e8751`, `3d9645200bc5`, `e1e22d48db4d`, `f56cd4a0b6c9`, `e7fc93e0bb77`, `78ddf4d5ea27`, `9a4415ddafa6`). The new machinery exercised itself live in this run: `record-verify` on the record passes every recomputation — 52 evidence refs against their sidecars with 0 dispute-bearing findings lacking complete refs, `precedent_scope_valid` against the new 19-entry store, `fixture_scope_valid`, coverage — and fails only `gate_pass`, because the gate is BLOCKED. Record: `review-v110/codereview-20261105-001944.json`.
|
|
2130
|
+
- Suite: 390 tests (29 new on the merged line: `tests/test_role_providers.py`, `tests/test_user_setup.py`, incl. fresh-HOME no-defaults and non-interactive refusal coverage).
|
|
2131
|
+
|
|
2132
|
+
## 0.10.0 — 2026-11-04
|
|
2133
|
+
The integration release: the Step 4 machinery branch (`step4-ladder`, built in parallel in an isolated worktree) merges into the main line, its amendment review's two confirmed majors are fixed, and the refutation-process solution's build steps 1–2 land (owner-adopted design: `codereview-refutation-process-solution-2026-11-04.md`). No closing attempt ships in this release; the Tech Lead gains NO binding authority here — the shadow-mode upgrade, probe harness, and known-answer acceptance test are the next track. The announcement hold stays ON.
|
|
2134
|
+
- **Step 4 machinery merged (`ba4c888` + `15a3958`, merge `bbadad1`; textual merge clean, merged suite 338/338)**: refutation-adequacy routing (§5a.1 — a refutation whose validated quote merely repeats the cited line is graded `refutation-weak`, a routing state, not a verdict); the severity rubric (§5b) in the reviewer/verification prompts and policy (a major requires a demonstrated reachable defect in a supported configuration, with the reachable path cited; no silent downgrades anywhere); the supported-platform manifest (§5c — POSIX, Python ≥3.11, carried on every record and surfaced in the prompt; Windows-only findings deterministically labelled `out-of-scope (documented)`, visible and non-blocking); function-level precedent (§5d — precedent keys on the enclosing function's normalized AST hash plus direct callees, so unrelated edits to the same file no longer re-open settled rulings; legacy file-level entries, incl. the 5 seeded guards, keep working); and **Tech Lead adjudication** (owner amendment superseding the §5a.3 tiebreak): any cross-family refutation — disputed or refutation-weak — goes to a seat whose family neither raised nor refuted it, whose decision (sustain_finding, with rubric downgrade only / sustain_refutation / escalate) is the gate disposition, recorded with rationale, validated quote, family/tier, and all three positions. An ineligible designated seat (`--tech-lead`) is rejected and substituted loudly, never silently; with no independent seat the finding keeps its disputed/weak state for the owner. Sustained refutations and rubric downgrades are precedent-eligible (`precedent_candidates` on the record; persisting stays the caller's step — a run never mutates the store it is verified against).
|
|
2135
|
+
- **Amendment review fix (a) — Tech Lead call timeout guard (confirmed major `1321bec43a24`)**: adjudication calls ran synchronously inside `run_review` with no wait bound; a hanging seat could block the whole pipeline. `_tech_lead_call` now runs under `_review_job_bounded` (daemon worker thread, hard wait bound, default `TECH_LEAD_CALL_TIMEOUT = 900`s — the panel's per-seat ceiling); `run_review(tech_lead_timeout=...)` threads it and the CLI wires `--timeout` through. A timeout returns `invalid` and the case escalates on the record with the timeout named; the bound in force is recorded per adjudication (`call_timeout`).
|
|
2136
|
+
- **Amendment review fix (b) — explicit probe-loop bound (confirmed major `f6064fc5e515`)**: the one-probe-round limit was previously an accident of the code's shape. It is now an explicit counter against `MAX_PROBE_ROUNDS = 1` in a loop; a further `needs_probe` escalates with the bound named, and `probe_rounds` is recorded per adjudication.
|
|
2137
|
+
- **Archived-record exclusion (solution §5.4 / build step 2)**: committed review output (`review-*/` directories) and the `gate-attempts.*` ledger are generated artifacts, not source. Their diff sections are excluded from the reviewed span before batching — reviewers never see them — and every exclusion is listed in the record's `archived_exclusions` with the rule that matched, plus a gate reason. A finding that nevertheless cites archived material (its path is an archived artifact, or its quote exists ONLY inside archived sections — the re-raise shape that produced two of the v0.9.6 review's disputed majors) classifies as archived-text in `archived_text_findings`: preserved, non-blocking, never adjudicated. Verification rounds classify their new findings by the same rule. This kills the re-raise class mechanically, upstream of the whole dispute ladder.
|
|
2138
|
+
- **Evidence sidecars (solution §4 Q7 / build step 1)**: evidence text is no longer truncated at ingest — finding evidence/quotes, re-ground and cross-family quotes/reasons, Tech Lead outputs, probe records, deterministic-scan evidence, and verification-resolution evidence are all stored full-text (prompt construction slices for display only, in `_finding_public` and the prompt templates). At record-write time, `write_report` stores every such text as a content-addressed sidecar (`<out>/evidence/<sha256>`) and the JSON keeps a display copy (400/300/600-char caps by field) with a `{sha256, length, truncated}` ref per text. Dispute-bearing findings are stamped `evidence_complete` when every position text carries a ref — the flag the Q2 eligibility checklist reads: an adjudication over incomplete inputs is record-ineligible. `record-verify` gains the `evidence_sidecars` check (and `record-verify --chain` its chain-wide form): refs must recompute — sidecar present, hash and length matching, display copy an exact prefix, truncated flag consistent — or the record FAILS; a record with no refs (pre-0.10) reports not-applicable, so legacy records verify exactly as before.
|
|
2139
|
+
- **Protected-class residue of the branch's own reviews**: the step4 branch's internal reviews left 4 disputed majors (`bb3deba73d59`; `aecd8d6b5f57`, `85ac395128cd`, `24929379e396`), all about the adjudication/precedent machinery itself — a protected class under the adopted design (Q3: the Tech Lead never finally decides its own machinery). They are recorded as disputed, NOT adjudicated and NOT routed to any Tech Lead, and are compiled for the owner's ruling with both positions and code evidence in the release report. The same reviews' one standing minor (`--tech-lead` flag validation, `d4423cae53bf`) stands as received; designation misuse is already rejected loudly at adjudication time by `_select_tech_lead`.
|
|
2140
|
+
- **Internal review of this diff (verdict as received — no PASS is claimed)**: **BLOCKED**, not degraded, coverage complete, the record's own test run green (361 tests); 5 findings withdrawn in the re-ground pass. Gemini quota-failed on every batch (not retried); the OpenAI fallback served correctness, NVIDIA operations. Standing findings: one deterministic **blocker** — the eval/exec rule firing on the literal `eval(preserved_input)` string inside the new test file's archived-record fixture (a true positive against fixture text, the class the fixture-manifest mechanism exists to classify; it stands as received) — and one minor (the uncapped test-failure `output_tail` in deterministic evidence may carry large outputs). One major stands **disputed** (`4ce0212286e0`, the `_finding_public` prompt-slicing claim) and two majors **refutation-weak** (`6b29fe953939`, function-level precedent callee collision; `bf4c451e8751`, a `write_report` robustness claim) — all three concern the evidence/precedent machinery itself (protected class), no independent Tech Lead served (Gemini was down), so all three are recorded exactly as received for the owner gate; the builder adjudicates none of them. The new machinery exercised itself live in this run: the record carries 21 evidence sidecar files (42 refs, all recomputing under `record-verify`'s new `evidence_sidecars` check, which passes), and the span contained no archived artifacts to exclude. Record: `review-v100/codereview-20261105-000409.json`.
|
|
2141
|
+
- Suite: 361 tests (23 new in `tests/test_v100_integration.py`).
|
|
2142
|
+
|
|
2143
|
+
## 0.9.6 — 2026-11-04
|
|
2144
|
+
The probe-pack release of the v15-closing-run solution (§4, executed as Step 2 of §6): every disputed major from review-v15 that the solution assigned a probe now has that probe in the suite permanently (`tests/test_v096_probes.py`), so the owner's §3 rulings rest on rerunnable evidence. This release records NO rulings and runs NO closing attempt. Under the owner's adopted §2 rule ("owner rules, builder supplies evidence"), the two v0.9.5 fix-diff disputes are owner-list items #15/#16 (addendum to the v15 closing-run report); the builder adjudication in the 0.9.5 entry below stands as history, superseded as a ruling.
|
|
2145
|
+
- **Probe results** (probe → disputed finding → outcome): P1 (3c39c4bdc15a) PASS — a free-tier seat behind a local stub endpoint is refused under `--private` before any call (the stub saw zero requests; the control run without `--private` reached it), and a record attesting `private=true` with a successful free-tier seat fails record-verify's `tier_policy` check. P3 (474297e6ffdc) PASS — a forced bench rerun's in-memory records contain none of the stored degraded records (marker cost/timestamp absent; the file is replaced, and the report's `degraded_cases` is 0), and a rerun dying mid-run leaves the previous results file byte-identical (the v12 safety). P4 (09c536d15270) PASS — the doc's property test verbatim in spirit: n 1..59 × pct 0..100 never raises, always returns a member, pct=100 returns the maximum and pct=0 the minimum. P5 (bca4074a632c) PASS — statically, every read of the attempts-log setting is `args.attempts_log` (argv only; nothing assigns it, no environment lookup feeds it, no other module references it), and the flag's help text now states that trust level next to it. P10 (c5f02551d077) PASS with the directed label fix below — a solo verification round is labelled in every outcome, and a chain containing one still fails `rounds_two_families`. P11 (0e57186e8be8) PASS — `agent:x:sh -c 'echo a:b'` parses with the command intact, and the built seat executes it. P12/P13 (1aec997aaec4 + 0ad4ed7db98e) PASS — the doc's property test over all 13,692 permutations of subsets of the mixed spec list (a preset, agent case variants, agent suffix collisions, a custom API spec): every combination either raises `ProviderError` loudly or builds exactly one seat per spec with case-insensitively unique names; no silent merge anywhere in the space. P14 (2242edbba9c9) PASS — providers' module scope provides the `os`/`signal` objects `_kill_tree` uses, and the module imports clean under warnings-as-errors.
|
|
2146
|
+
- **P9 (d93089253022) — the matrix confirms the overmatch refutation, and surfaces one thing §3 did not predict**: patterns `app/` and `app` match exactly the directory's children (the sibling `app/users2.py` and the prefixes `app2/`, `application/` never match), and `app/users` rejects its prefix traps — but the spelling `./app` matches NOTHING, not even the directory's own children: the leading `./` is not normalized away, so a manifest author writing it gets an inert pattern. The direction is fail-safe (an inert pattern marks fewer fixtures; it can never hide a product finding), but it is silent. Pinned by a clearly-named characterization test; per the adopted rule this goes back to the owner WITH the §3 ruling rather than being silently fixed here.
|
|
2147
|
+
- **The shared `_is_within` helper (P6/P8 — eae8137a6db3 + 3cecf66f6ce5; the fix §3 directs at minor severity)**: both containment call sites (the fixture-manifest loader and the precedent loader, both in `cli.py`) now share one helper that realpaths both sides, compares with `os.path.commonpath`, and returns `False` on `ValueError` instead of propagating it — a containment question that cannot be evaluated fails closed. Tests pin the monkeypatched-`ValueError` path (helper `False`; the loaders raise their own "resolves outside" error), the `ntpath` different-drives `ValueError` that is the mechanism behind the findings, and unchanged within/outside behaviour including the prefix-sibling case.
|
|
2148
|
+
- **Solo-panel label in verification records (P10's directed fix, c5f02551d077)**: `run_verification` appends the standalone "solo panel — no independent review" reason whenever the verification panel has one seat or fewer. Previously only a clean solo PASS carried the wording (embedded in its pass sentence), so a solo round ending blocked or inconclusive read like a panel verdict. The review path's label is unchanged, and eligibility still fails a solo round at `verify_chain`'s `rounds_two_families` check.
|
|
2149
|
+
- **Internal review of this diff (verdict as received — no PASS is claimed)**: **BLOCKED**, 88 standing findings (20 blocker, 40 major, 28 minor), not degraded, coverage complete, the record's own test run green (299 tests); 29 further findings were withdrawn in the re-ground pass. Gemini quota-failed on every batch (not retried) and the OpenAI fallback served correctness, with NVIDIA on operations. Composition, stated plainly: the release commit also newly tracked the prior rounds' record files (review-v9c…v15, review-v095-fix, `gate-attempts.jsonl`, ~13.6k lines of historical evidence), and 68 of the 88 findings target those artifacts — including findings about code as preserved inside the old records; the 20 findings on source files sit overwhelmingly in pre-existing code outside this release's hunks. 6 findings were refuted cross-family and are recorded as **disputed, not adjudicated** (per the adopted §2 rule): two of them re-raise the P6/P8 `commonpath` claims against the pre-fix lines preserved in the historical records. The verdict stands as received; disposition of its findings belongs to the owner's ruling process and later steps, not to this release.
|
|
2150
|
+
- Suite: 299 tests (23 new).
|
|
2151
|
+
|
|
2152
|
+
## 0.9.5 — 2026-11-04
|
|
2153
|
+
The review-v15 machinery release: the three build items the Phase 3 closing run's successor report (`2026-11-04-codereview-v15-closing-run.md`) named as the bars to any further closing run. No closing attempt ships in this release — the re-run also waits on the owner's rulings on the 14 disputed majors, which this release does not touch, waive, or adjudicate.
|
|
2154
|
+
- **Oversized diff sections are sub-split, never truncated (bar 1)**: `_batch_sections_detailed` cut a section larger than the batch budget down to size (`text[:max_chars]`), which is why review-v15's round 1 degraded with incomplete coverage — `src/codereview/core.py`'s cumulative section is 94,494 chars against the 80,000 budget, while file/lens omissions were 0. A new `_split_section` now sub-splits an oversized section at diff-hunk boundaries, then at line boundaries inside a single oversized hunk, then by character-splitting a single over-long line; every piece repeats the section head (and the hunk header on line-split pieces) so a reviewer always knows which file and hunk a piece belongs to, hunk bodies are never dropped or reordered, and the coverage ledger names the file in every batch that carries a piece of it. The batching flag (record field `truncated`, kept for compatibility) now means batching *incomplete* — content that could not be placed within the budget at all (only reachable when the budget cannot hold even a minimally framed slice) — and the run still degrades loudly in that case; the gate and coverage reason strings say so in those words. Regression tests cover a multi-hunk section landing completely, the v15 shape at its real numbers (~94.5k section, 80k budget), line-boundary and character splits, the unplaceable edge, and a full `run_review` whose coverage stays complete over an oversized section; the two tests that pinned the old truncate-and-flag behaviour are updated to the new semantics.
|
|
2155
|
+
- **Verification rounds honor the fallback seat map (bar 2)**: in review-v15-verify1 the panel was quota-failed Gemini + NVIDIA only — the runner supplies a fallback map (`fallbacks_override`, OpenAI behind the correctness lens) but the CLI's verify branch never forwarded it and `run_verification` did not accept one, so the one OpenAI-raised blocker could not be finder-confirmed and came back unjudged. `run_verification` now takes the same lens → seat `fallbacks` map as `run_review`: when a primary fails, the fallback configured for the lens that seat would serve in a review round (the `_reviewer_tasks` assignment) runs the verification job under the same bounded-substitution contract as a review round — reason recorded in the gate reasons, only the job budget's remaining attempts spent, the report carrying the serving seat as `provider` with `fallback_for` and `primary_error`, new findings attributed to the serving seat — and the CLI forwards the map in verify mode, so the runner's verification path is wired end to end. Review-mode behaviour is unchanged, as are the loop and PR paths (they pass no map). Regression tests pin the deadlock's resolution (quota-failed primary → fallback serves → finder confirmation completes, gate PASS), the no-map behaviour (finding stays unjudged, INCONCLUSIVE), a failing fallback recorded on the primary's report, and the CLI seam the runner actually uses.
|
|
2156
|
+
- **Chain verification normalizes abbreviated SHAs (bar 3)**: `record-verify --chain` compared range endpoints to `head` fields as raw strings; records store full SHAs in `head` while the v15 ranges were supplied abbreviated (`e3fc6a8` vs `e3fc6a8c12ba…`, the same commit), so `ranges_contiguous` reported a gap in a contiguous chain. Both sides are now resolved to full SHAs against the repo (cached per run) before comparison; an endpoint that does not resolve compares as its raw string, so a bad ref still fails honestly, and a genuinely gapped chain — abbreviated or not — still fails. Regression tests verify an abbreviated chain identically to its full-SHA form and keep an abbreviated gap failing.
|
|
2157
|
+
- **Internal review of this diff (verdict as received — no PASS is claimed)**: **INCONCLUSIVE**, 0 standing findings, not degraded, coverage complete, the record's own test run green (276 tests). Gemini quota-failed (not retried) and the OpenAI fallback served correctness — the fallback machinery working in its new verify-adjacent shape — with NVIDIA on operations. Two majors were refuted cross-family and routed to the human gate as disputed; the builder adjudicates both as false positives on code evidence: (1) "an unplaceable oversized piece is appended to a fresh batch without any size guard" describes the designed loud path, not a defect — the only alternative to carrying unplaceable content whole in its own batch is dropping it, and the batching flags the case (`incomplete` → coverage incomplete → run degraded, stated in the gate reasons; pinned by `test_unplaceable_budget_is_loud_and_keeps_content`); (2) "the verification fallback lens mapping is wrong for the second seat" hypothesizes a panel with more than two lenses, but `LENSES` is a two-entry constant (correctness, operations) and the mapping copies `_reviewer_tasks`' assignment verbatim — the same assignment the review path itself uses, which is precisely the parity bar 2 set.
|
|
2158
|
+
- Suite: 276 tests (12 new).
|
|
2159
|
+
|
|
2160
|
+
## 0.9.4 — 2026-11-04
|
|
2161
|
+
The review-v15 adjudication release — the Phase 3 closing run's round 1 and its verification, shipped with verdicts exactly as received. Round 1 (attempt 11, span `3013656..e3fc6a8`, record `review-v15/codereview-20261104-224759.json`) came back **BLOCKED and degraded**, so this release changes no product code: all five round-1 blockers were rebutted on executable evidence, now pinned as characterization tests in `tests/test_v094_probes.py` (suite 264/264) — the review fallback construction does forward `agent_timeout` (cli.py:698-699); every provider call path carries a timeout (`_post_json`'s `DEFAULT_TIMEOUT = 240` is the only opener site); `_call_with_retry` is hard-bounded per class and per job; `_classify_error`'s whole-token matching is the v0.9.3 fix itself; and `_kill_tree` cannot flip a successful agent run (it never raises on a dead group, and the return code is captured first). The new machinery did its job in the open: 9 findings withdrawn by their own seats in the re-ground pass, 5 standing findings auto-linked under precedent, 44 fixture findings classified. Two things keep the record ineligible, stated plainly: **coverage was incomplete** — `src/codereview/core.py`'s cumulative diff section is 94,494 chars, over the 80,000-char batch budget, and `_batch_sections_detailed` truncates an oversized section instead of sub-splitting it (0 file/lens omissions; the sub-split fix is the next round's first build item, and it needs a fresh round 1); and **14 majors stand disputed** — refuted cross-family with valid quotes, therefore out of the blocking set but not cleared, awaiting the owner's human-gate ruling. Full dispositions: `review-v15-dispositions.md`. The announcement hold stays on.
|
|
2162
|
+
|
|
2163
|
+
## 0.9.3 — 2026-11-04
|
|
2164
|
+
The Phase 2 release of the owner's v14-hold solution — the release that stops the clean-gate treadmill instead of feeding it another round. Phase 0 of that plan reconstructed review-v14's coverage from the record and **ruled truncation out**: all 430,878 post-redaction patch chars reached both lenses (0 omitted hunks), and the two findings that contradicted verified facts cited text that WAS in the batches — the failure was reviewer quality on large batches (a seat counted the corpus's 9 holdout cases twice: 36 + 9 = "45"), not missing text. So this release's centre of gravity is grounding and confirmation, with coverage kept as the record's own proof. It also carries the Phase 1 adjudication fixes (commit `5119998`, previously unreleased): 2 real defects fixed, 13 findings rebutted with executable evidence — full table in `review-v14-dispositions.md`, coverage reconstruction in `review-v14/coverage.md`. The owner adopted all three of the solution's reserved decisions on 2026-11-04 — Form B, record-and-attest, content-addressed precedent — and all three are built here. No closing attempt ships in this release; Phase 3 runs it.
|
|
2165
|
+
- **Evidence-grounded findings with an automatic re-ground pass (solution §2b)**: the review prompt now requires an `evidence_quote` copied exactly from the diff, and every model finding's quote is validated deterministically against the batch text its reviewer actually received. A failing finding is never dropped: the raising seat (or another family's) is shown the exact file section and must confirm with a valid quote or withdraw. Confirmed → the finding stands, marked `reground_confirmed` with the validated quote; withdrawn → preserved in `withdrawn_findings` with the seat's reason; unresolved → the finding stays visible and a would-be PASS becomes INCONCLUSIVE for a human. Deterministic findings are validated against the material they cite (or marked not-applicable, e.g. test-output evidence).
|
|
2166
|
+
- **Cross-family confirmation for blockers (solution §2c)**: a blocker/major raised by one vendor family is put to a different family's seat that served the run, with the cited file's section as the region. Confirm → it stands (`cross_family: confirmed`). Refute counts only with a quote that validates against the region → the finding becomes **disputed**: out of the blocking set, listed in `disputed_findings`, and the gate may not auto-PASS (INCONCLUSIVE — the human gate decides; it is neither auto-cleared nor silently blocking). A refutation without a valid quote, a failed call, or no second family served leaves the finding standing, marked `unresolved`/`unavailable`.
|
|
2167
|
+
- **Coverage accounting carried by the record (solution §2a)**: every review record now carries per-file, per-lens batch coverage derived from the actual batching (each reviewer report names its batch). A file/lens omission — or a truncated section — marks the run degraded at review time and is stated in the gate reasons. `record-verify` gains `coverage_complete`: a record WITH coverage data is held to it; a record without (pre-0.9.3) reports the check not-applicable rather than failing retroactively.
|
|
2168
|
+
- **Form B chain verification (solution §2d)**: `codereview record-verify --chain <round-1> <verify> [...]` verifies a round-structured record as one object — ranges contiguous base→HEAD (hash-checked, no gaps), every round non-degraded with two families, round-1 lens coverage and complete coverage, every round-1 blocker/major dispositioned on the record (present in a verification round's `cleared` list — an all-`invalid` clearance is the ruled rebuttal), final gate PASS, tests passing, final head current (later commits may touch only the chain's record directories). The adopted convergence budget (≤3 verification rounds, non-increasing new-blocker counts) is emitted as `[WARN]` lines: a process flag for the owner, distinctly NOT an eligibility failure. Verification records now also carry `test_results`, `range`/`head` (via `record_extra`), and the `private` attestation, so chains can be built from real CLI runs.
|
|
2169
|
+
- **Tier attestation — record-and-attest (owner decision (a), settling v13 finding [4])**: run records carry the run's `private` declaration; seats already record their tier. `record-verify` gains `tier_policy`, which fails ONLY the adopted conjunction — `private=true` AND a successful (ok-status) seat on a free tier. A public run attests `private=false` and passes with free seats; a record with no declaration reports not-applicable. The run-time refusal in `--private` (free seats refused before the run) is unchanged.
|
|
2170
|
+
- **Content-addressed rebuttal precedent (solution §2e, owner decision (b))**: a new store, `precedents.json` (schema `codereview.precedents/1`, passed with `--precedents`), keys finding fingerprint + cited path → the cited file's git blob at disposition time, the disposition reference, and the rounds it was seen in. During record assembly a matching finding whose cited file's blob at HEAD is unchanged is classified `standing` — listed in `standing_findings` with its repeat count and disposition link, non-blocking; a changed blob means fresh adjudication, so a precedent can never hide new code. `record-verify` recomputes the store hash and every link (entry exists, blobs equal at the recorded head) exactly as it validates fixture scope. The store is seeded with the five guard-minor dispositions sustained across review-v11..v14 (the POSIX platform-guard repeats); every seeded file is unchanged since its single touching commit, so HEAD blobs are the disposition blobs (basis recorded per entry). `core.upsert_precedent` maintains entries programmatically.
|
|
2171
|
+
- **Fixture expected counts (solution §2f)**: a manifest entry may declare `expected_finding_count`; findings on that path beyond the count are NOT classified as expected — they remain product findings in the record, the excess is noted in the gate reasons and in `fixture_scope.excess`, and record-verify's recompute stays consistent (the count classifies, the excess never disappears). The shipped manifest declares no counts (policy-v1 behaviour unchanged).
|
|
2172
|
+
- **Phase 1 fixes riding this release (review-v14 adjudication)**: `_classify_error` matched HTTP status codes as bare substrings, so a 503 body mentioning "4030 tokens" (or a request id "req_4014") classified as auth and a transient failure was denied its retries — status codes now match as whole tokens, and the typed provider quota codes (`resource_exhausted`, `insufficient_quota`) decide the quota class directly. And bench `--force` now refuses to swap in a results file when any case in the forced run came back degraded: `run_review` absorbs provider failures into degraded records rather than raising, so a forced run during an outage silently replaced good results with 27/27 degraded ones (probe-verified); the refusal keeps the previous file byte-intact and exits cleanly with the reason.
|
|
2173
|
+
- **Housekeeping (solution §2f)**: `codereview doctor` output now states the live-probe rule in the result itself — a seat counts as wired only after a live probe in the runner's exact review call shape (the probe prompt already uses the real review shape; verified, no residual toy probe in any path). `gate-attempts.md` already carries the attempt-6, review-v11 and attempt-10 rows (completed in the v0.9.0/v0.9.2/Phase-1 rounds).
|
|
2174
|
+
- **Internal review of this diff (two rounds, verdicts as received — no PASS is claimed)**: round 1 returned **INCONCLUSIVE** under the new machinery itself — 0 standing blockers, coverage complete, not degraded; 13 findings were withdrawn by their own seats in the live re-ground pass (the §2b mechanism working on its own release), and 4 majors were refuted cross-family and adjudicated at the human gate as false positives or by-design, each verified against the code (the precedent loader canonicalizes `repo_abs` exactly like the fixture loader; the missing-blob scenario is guarded in `apply_precedents` and at load; deterministic grounding is validated in `run_review`; the chain's test-results requirement is the documented Form B design). Of its 4 minors, one (direct unit coverage for `_batch_sections_detailed`) was fixed in-release and three rebutted (two are the deterministic guard-text hits on quoted text; one asked to verify a constant that is defined and test-covered). Round 2, on the updated diff, returned **BLOCKED** on two majors claiming the new logic "lacks unit tests" — both rebutted with the test classes in the same diff (the claim is a per-batch blindness artifact: the seat judging a batch cannot see the test file in another batch, while the record's own test run executed all 253) — plus 4 more disputes, all sustained on schema/code evidence, and 5 minors rebutted. No real defect was found in either round; the rounds are recorded here because the saga's rule is that verdicts are reported as received.
|
|
2175
|
+
- Suite: 253 tests (37 new).
|
|
2176
|
+
|
|
2177
|
+
## 0.9.2 — 2026-11-04
|
|
2178
|
+
The review-v13 adjudication release. The v0.9.1 closing attempt (review-v13) blocked on 6 majors + 5 minors + 1 nit; every one was adjudicated against the code — 1 fixed as a real defect, 4 rebutted as false positives with executable evidence (one of them, the "unused import" nit, refuted by the very file it cited), 1 left as an explicitly owner-reserved decision with a decision memo (no builder accept, no silent enforcement change), and the 5 minors recorded as standing repeats of a rebuttal first sustained in v11. The full disposition table, with finding IDs and evidence, is `review-v13-dispositions.md`; the governing issue report is `~/workspace/issue-reports/2026-11-04-codereview-v13-gate-findings.md` (outside the repo).
|
|
2179
|
+
- **Explicit agent timeouts are honored or refused — never silently swapped (real, v13 [5])**: `build_provider` built agent seats with `timeout=agent_timeout or AGENT_TIMEOUT`, so a library caller's explicit `agent_timeout=0` became the 900-second default without a word (proven by probe: 0 → 900 captured; a negative value passed through to fail late and confusingly). The CLI boundary has rejected non-positive `--timeout` since 0.9.1, but the library seam had not. `build_provider` now passes `AGENT_TIMEOUT` only when `agent_timeout is None`, and `command_agent` — its sole construction path — refuses a non-positive timeout at construction with a `ProviderError`. Regression tests pin all four quadrants (None / zero / negative / positive) at the singular, plural, and direct seams (`tests/test_v092_fixes.py::AgentTimeoutSeamTests`).
|
|
2180
|
+
- **Characterization tests pin two rebutted invariants** so they cannot rot silently: bench resume after a config rename (v13 [2] — the config name is inside `_config_sha`'s hashed identity, so foreign records are dropped as stale before dedup; the test drives the strongest form, old-name records planted in the renamed config's own results file) and fixture sibling-prefix rejection (v13 [3] — every pattern in the shipped `fixtures-manifest.json` is swept against sibling mutations; matching stays boundary-aware in both pattern branches).
|
|
2181
|
+
- **Owner-reserved, deliberately untouched**: whether `record-verify` should enforce the free-tier policy (v13 [4]) is a decision about what a clean record claims, not a code defect this round may settle; `review-v13-dispositions.md` carries the decision memo (current policy mechanics, the missing `private` declaration in the record schema, three options, a recommendation). The rebuttal-precedent mechanism proposed in `review-v12-dispositions.md` likewise remains the owner's decision — the five guard minors in v13 are its standing repeats.
|
|
2182
|
+
- Suite: 202 tests (9 new).
|
|
2183
|
+
|
|
2184
|
+
## 0.9.1 — 2026-11-04
|
|
2185
|
+
The review-v12 adjudication release. The v0.9.0 closing attempt (review-v12) blocked on 10 product findings + 6 minors; every one was adjudicated against the code — 3 fixed as real, 6 rebutted as false positives with executable evidence, 1 recorded as a repeat of an already-sustained rebuttal, and 1 code-reading correction to the round's own triage (the doctor half of the "confirmed" timeout pair already forwarded at HEAD; the enumeration test now proves every path instead of trusting the claim either way). The full disposition table, with finding IDs and evidence, is `review-v12-dispositions.md`; the governing issue report is `~/workspace/issue-reports/2026-11-04-codereview-v12-gate-findings.md` (outside the repo).
|
|
2186
|
+
- **Review runs can configure the agent timeout, forwarded to primaries AND fallbacks (real, v12 [3])**: the review command had no `--timeout` option at all, and its fallback construction omitted the forwarding that doctor/intake already had — an asymmetry waiting to bite. `codereview --timeout N` now exists and reaches both constructions in `main()`, exactly mirroring `doctor --timeout`. The doctor fallback path (v12 [4]) already forwarded at HEAD — git history shows it landed with v0.9.0 and its evidence quote showed the forwarding lines; what was missing was a test exercising the fallback branch, which the enumeration suite below now supplies for every path.
|
|
2187
|
+
- **Enumeration test over every seat-construction path (closure criterion)**: `tests/test_v091_fixes.py` walks the AST of `cli.py`, `bench.py`, and `providers.py` and accounts for EVERY `build_providers`/`build_solvers` call site: each either forwards `agent_timeout` or belongs to a command that exposes no timeout option (loop / workflow / pr / bench-config), with a companion assertion that an allowlisted command still has no `--timeout` flag — so adding a timeout option anywhere without wiring its forwarding fails the suite loudly, as does adding a new construction path without accounting for it. Functional spy tests drive review and doctor with `--timeout 7` and assert primaries and fallbacks both receive it.
|
|
2188
|
+
- **Bench `--force` replaces stored records instead of duplicating them (real, v12 [1])**: a forced rerun skipped the staleness filter but still appended, so each forced run stacked a second set of records for the same (config, case) into the results file. Replacements now stream to a sibling temp file and are swapped in atomically when the config's run completes, so a mid-run failure also leaves the previous results intact (hardening from this release's own internal review round); regression tests cover both the record count and the failure case.
|
|
2189
|
+
- **Non-positive `--timeout` values are rejected at the CLI boundary** (internal review of this fix diff): review, doctor, and intake validate the flag before any seat is built — a zero or negative value would otherwise reach the agent subprocess and surface as a confusing seat error. Regression tests cover all three commands.
|
|
2190
|
+
- **Fixture classification hardened at the source (v12 [8])**: `_fixture_inner_paths` now subtracts the patch's top-level file names itself, instead of relying on each caller to remember the check. A product file can no longer be classified as a fixture inner path even by a future caller that forgets the guard; the callers' own checks remain as defence in depth. Unit tests cover the colliding and non-colliding shapes.
|
|
2191
|
+
- Suite: 193 tests (18 new).
|
|
2192
|
+
|
|
2193
|
+
## 0.9.0 — 2026-11-04
|
|
2194
|
+
The review-v11 adjudication release, built to the owner's clean-gate solution. The v0.8.0 span review (the first attempt where the OpenAI fallback served the correctness lens end-to-end) blocked with 40 findings; every one was adjudicated against the code — 26 were the benchmark's own fixture data, 4 were real, 9 were false positives, 1 minor was a docs gap. The full disposition table, with finding IDs and evidence, is `review-v11-dispositions.md`.
|
|
2195
|
+
- **Fixture scope as a versioned, machine-checked manifest (solution §3)**: `fixtures-manifest.json` (policy v1) enumerates every fixture path — all 36 corpus cases and the three scripted-seat response files — with the reason each is intentionally vulnerable and its expected finding categories. A release-record review passes `--fixture-manifest`: the patch is still reviewed **in full** (nothing is dropped), but findings belonging to declared fixture data are classified in the record as `fixture_findings` — evidence and counts preserved — and do not block the gate. The record carries the manifest path, its content hash, the policy version, and the classified paths; `record-verify` recomputes all of it against the manifest committed at the recorded head and FAILS on a missing, stale, or mismatched manifest, on classified paths that differ from the manifest, or on a fixture finding the manifest does not cover. Findings reported under a fixture's *inner* paths (a case diff's own little file tree) are classified too — unless the same path is also a real file in the change, in which case the finding stays blocking. Ordinary reviews without a declared scope scan fixtures exactly as before.
|
|
2196
|
+
- **Doctor probes use the exact review request shape (solution §4)**: the preflight probe is now the real review prompt over a trivial diff. The old toy probe ("reply OK") had produced a documented false-negative for a healthy JSON-mode seat, whose provider rejects non-review-shaped requests. `codereview doctor` also probes **fallback seats** (a fallback counts as healthy coverage only after this live probe; `--fallback lens=spec`, or the runner's configured fallback) and forwards `--timeout` to provider construction like review and intake do. Per-seat error classes still distinguish quota/transient from missing credentials (auth).
|
|
2197
|
+
- **Bounded primary+fallback attempts (solution §4)**: one reviewer job has a hard budget of 6 provider calls; the fallback job is handed only the remaining budget after the primary's attempts, and if the budget is already exhausted the fallback is not attempted — with the reason recorded in the gate reasons. A fallback-served report's attempt count is now the primary+fallback total (it previously recorded only the fallback's, understating run cost — a real v11 finding).
|
|
2198
|
+
- **Bench resume no longer trusts stale records (real)**: stored result records are re-validated on resume — the recorded patch hash must match the case's current diff and the recorded config hash must match the current config; stale records are dropped and the results file rewritten. `bench report` likewise drops hash-mismatched records and counts them (`stale_records_dropped`) instead of scoring findings gathered against a different change. Records now carry `config_sha256`.
|
|
2199
|
+
- **Seat-name collision checks are case-canonical (real)**: the cross-kind check (agent seat vs API seat) compared names case-sensitively on the custom-spec branch, so a case variant could slip an agent seat past an API seat of the same name. All pre-pass comparisons, including suffix bookkeeping, now use the lowercased canonical name.
|
|
2200
|
+
- **Default seat metadata reads the seat kind, not the name (real)**: resolved agent seat names are bare and suffixed (bot, bot-2), and one named like a preset inherited the preset's vendor family. `build_providers` now tags agent callables with their kind; `_default_seat_meta` reads the tag, so agent seats record family `agent` under every name.
|
|
2201
|
+
- Docs: the bench growth path names an owner and a timeline; the ROADMAP's fixture item is updated (manifest classification shipped; in-code suppression comments remain open).
|
|
2202
|
+
- Suite: 175 tests (40 new).
|
|
2203
|
+
|
|
2204
|
+
## 0.8.0 — 2026-11-04
|
|
2205
|
+
Epic A — the benchmark harness and corpus v0 (the revised plan's Week-1 epic; it blocks everything else, so it ships before any new surface):
|
|
2206
|
+
- **`codereview bench run|report`**: run named panel configurations over a labelled corpus and score them against ground truth. `bench run --config X` (repeatable; configs are small JSON files naming provider specs and/or seats) appends per-case records to `bench-out/<config>.jsonl` and resumes — recorded cases are skipped unless `--force`. `bench report` prints Markdown and can write MD+JSON; `--max-escaped-rate R` turns it into a CI regression gate.
|
|
2207
|
+
- **Metrics, defined in code and in `bench/README.md`**: recall, precision, clean-case false-positive rate, cost per verdict, p50/p95 latency, and the headline **escaped-defect rate** — defects no finding blocked on (unseen, or reported only as minor/nit, so the default gate would have shipped them). Matching is file + location (line within 3, or title-token overlap when lines are far/missing); one-to-one, worst severity first.
|
|
2208
|
+
- **Corpus v0**: 36 cases (27 public / 9 holdout), 29 defect cases with 31 ground-truth defects + 7 clean controls — authored defect injections (secrets, SQL/shell injection, auth removal and bypass, dollars-vs-cents and partial-refund payments, races, swallowed errors, traversal, XSS) plus cases converted from this repo's own stored run artifacts (the demo health-checker change and its verified fix). Loader validates every case and refuses a manifest that disagrees with the directories on disk. Growth path to 50–100 documented in `bench/README.md`.
|
|
2209
|
+
- **Holdout discipline (A5)**: `bench report --public` drops holdout records before scoring — no holdout id, row, or number can appear in public output; a regression test asserts it.
|
|
2210
|
+
- **Scripted replay seats** for deterministic, zero-call CI runs (`oracle` replays ground truth, `blind` reports nothing, `noisy` complains about everything). Their scores validate the harness, never a model. On corpus v0 the scripted runs already surface two honest facts: the deterministic layer alone accounts for recall 0.452 (13 of 31 defects), and cross-layer duplicate findings under dissimilar titles cost precision — recorded in `bench/README.md`, not tuned away.
|
|
2211
|
+
- Suite: 136 tests (15 new).
|
|
2212
|
+
|
|
2213
|
+
## 0.7.3 — 2026-11-04
|
|
2214
|
+
The §4 hardening from the owner's clean-record solution — written after five attempts at a two-reviewer release record for the v0.7.1+v0.7.2 span failed on capacity and credentials, never on findings:
|
|
2215
|
+
- **Preflight** (§4.1): `codereview doctor` probes every configured seat with one minimal call and reports per-seat health as JSON (ok / error class / message). A scheduler aborts on a dead seat before a doomed run creates a record; a missing key surfaces as class `auth`, naming the actual problem.
|
|
2216
|
+
- **Error-class-aware retries** (§4.2): failures classify as quota / rate_limit / transient / auth / provider (typed, via v0.7.1's `retryable` flag, which wins over string matching) / unknown. Daily-quota and auth failures are never retried inside a run; per-minute 429s and 5xx/overload/timeouts get bounded backoff (3 tries). Attempts and error classes are recorded on every reviewer report — retries are visible, never silent. (Suite note: failing-seat tests now exercise real short backoffs; the suite takes ~25s.)
|
|
2217
|
+
- **Fallback seat per lens** (§4.3): `--fallback lens=provider-spec` (or a runner-supplied mapping) serves a lens when its primary seat fails after retries. The report records `fallback_for` + the primary's error, the gate reasons name the substitution, and Markdown prints it. A fallback cannot launder a panel: record verification (below) still requires two distinct vendor families.
|
|
2218
|
+
- **Machine-checked clean record** (§4.4): `codereview record-verify <dir> [--repo .]` returns eligible / not-eligible with per-check reasons: gate PASS, not degraded, tests passed, every lens served, ≥2 distinct known vendor families, the recorded range's patch hash recomputes identically, and no commits after the recorded head except ones filing the record directory itself. Records now carry `range` + `head` when produced from `--range`; pre-0.7.3 records fail the currency checks with the reason stated.
|
|
2219
|
+
- **Seat metadata + free-tier policy** (§4.6): reviewer reports carry `family` and `tier` per seat (`--seat-meta name:family[:tier]`; presets get family defaults, tier stays "unknown" unless tagged). `--private` refuses seats explicitly tagged tier `free` — free tiers may use submitted content to improve the provider's products, so they do not review private code.
|
|
2220
|
+
- **Attempt ledger** (§4.5): `--attempts-log FILE` appends one JSON line per run (mode, gate, degraded, range, seats with attempts/error classes/fallback use). This repo's own attempt history for the span is summarized in `gate-attempts.md`.
|
|
2221
|
+
- Internal runner: seats are tagged (Gemini google/free, NVIDIA nvidia/free), and an OpenAI seat (openai/paid) rides as the correctness fallback whenever its connector credential is registered (owner decision 2026-11-04). DeepSeek remains editorial-box-only and is never seated.
|
|
2222
|
+
- Suite: 120 tests (24 new).
|
|
2223
|
+
|
|
2224
|
+
## 0.7.2 — 2026-11-04
|
|
2225
|
+
External review of v0.7.1 (APPROVE the code; HOLD the announcement for a healthy two-reviewer gate record). Its two required items and all five recommended items are in:
|
|
2226
|
+
- **Unique agent names, always** (required, §3.1): final seat names are assigned in a pre-pass over all specs; suffixes bump the trailing number (bot → bot-2 → bot-3) until unique, so a literal `bot-2` seat can no longer merge with the second `bot` seat into a pool that never runs. All three seats in the reviewer's reproduction now run.
|
|
2227
|
+
- **Process group killed on every exit** (required, §3.2): the seat's group is killed after a normal exit too, not only on timeout — background children an agent spawns die with their seat.
|
|
2228
|
+
- **First-round lens coverage** (§3.4): a review PASS now requires every configured lens to have had a successful reviewer; a partially blind panel is INCONCLUSIVE, matching the verification standard. (Two older tests encoding the degraded-PASS behavior were updated to the new rule, with citations.)
|
|
2229
|
+
- **Write-side output cap** (§3.3): POSIX seats run under RLIMIT_FSIZE (8 MB per file); a runaway agent dies with SIGXFSZ and the seat reports the limit instead of filling the disk.
|
|
2230
|
+
- **Private-HOME seat option** (§3.5): `agent:<name>[home=private]:<command>` runs the seat with HOME set to its scratch directory, for agents that need no login state. Regression tests document the tension honestly: the default seat CAN read a planted key under HOME (subscription agents keep login state there); the private-HOME seat cannot. An obeying-stub injection test locks the scratch-cwd guarantee.
|
|
2231
|
+
- **Wording and labels** (§2, §3.6): the verification INCONCLUSIVE reason now says the way forward (fresh review with the finder present, or a human waiver); solo panels are labelled "solo panel — no independent review" in both review and verification pass reasons.
|
|
2232
|
+
- Gate record: tonight's runs are again degraded by the Gemini free-tier quota (the code is unchanged in the panel's eyes — NVIDIA clean); the publishable two-reviewer record for the 0.7.0→0.7.2 span is scheduled for ~03:20 ET, after the quota reset. Suite: 96 tests (12 new).
|
|
2233
|
+
|
|
2234
|
+
## 0.7.1 — 2026-11-04
|
|
2235
|
+
External review of v0.7.0 (APPROVE WITH REQUIRED FIXES): every required item verified against the code and fixed.
|
|
2236
|
+
- **Agent-seat environment is allow-listed** (finding 1): seats inherit only PATH/HOME/locale/proxy basics plus `CODEREVIEW_PROMPT`; provider API keys and tokens stay out unless the operator opts names in via `CODEREVIEW_AGENT_ENV`. Sentinel-key regression tests.
|
|
2237
|
+
- **Verification integrity** (finding 2): a PASS now requires a quorum of ≥2 successful reviewers when ≥2 are configured, and each prior blocker must be confirmed by the seat that raised it (deterministic findings clear only when the pattern no longer fires). Unconfirmed blockers are INCONCLUSIVE ("unjudged"), never a one-reviewer "consensus"; pass reasons state the real counts, and Markdown reports carry a ⚠ DEGRADED marker with the reviewer tally whenever a panel is degraded. (The shipped v0.7.0 verification record — PASS on one surviving reviewer — would be INCONCLUSIVE under these rules; a regression test encodes exactly that shape.) One deliberate refinement: a reviewer answering "unclear" still keeps the block (BLOCKED), as before — INCONCLUSIVE is reserved for panel-side inability to judge.
|
|
2238
|
+
- **Agent-seat isolation** (finding 3): seats run in a fresh scratch directory containing only the prompt file — never the repo working tree — and the README now states the boundary plainly: redaction covers the prompt CodeReview sends, not files a tool-enabled agent chooses to read; seat only agents you trust with their own permissions. The Claude Code example wrapper disables all built-in tools (`--tools ""`).
|
|
2239
|
+
- **Seat namespaces** (finding 4): an agent seat and an API seat sharing a name is now a loud configuration error in either spec order, instead of a silent pool merge.
|
|
2240
|
+
- **Lifetime** (findings 5, 6.1): agent seats run in their own process group; a timeout kills the whole tree. Solver seats default to the intake deadline (300s, or `--timeout`), so a hung agent can no longer keep the CLI alive past its run. Output is captured to files and capped at 1 MB (verdict tail kept). Agent-seat errors carry a typed `retryable=False`, so stderr text can no longer trigger account rotation (6.3). Noisy agent output (banners, fences) is covered by a parsing regression test (6.4).
|
|
2241
|
+
- Docs hygiene (6.7, 6.8): REVIEW-NOTES pointers corrected (the v0.1.0 `review-evidence/` package is not in this archive; gate records live in the `review-v*` folders); the 0.7.0 entry below is final. Suite: 84 tests (15 new).
|
|
2242
|
+
|
|
2243
|
+
## 0.7.0 — 2026-11-04
|
|
2244
|
+
Direction set by the revised plan adopted 2026-11-04: one promise — a verifiable verdict with a public record — proven by a public benchmark; launch scope narrows to CLI + Action + benchmark + run pages (console/iOS frozen as internal surfaces, hosted tier and editor extensions deferred to demonstrated demand).
|
|
2245
|
+
- **Agent seats** (`--provider "agent:<name>:<command>"`): any local agent CLI can sit on the reviewer panel or as an intake solver — Claude Code, Codex, a Grok bot, Gemini CLI, or a wrapper script — riding on the agent subscription the user already has; no vendor API key required. The prompt is delivered as a file (`$CODEREVIEW_PROMPT`) and on stdin; the agent's stdout is the verdict. The command line is shlex-split and run **without a shell** (metacharacters are literal; pipes/redirects belong in a wrapper script), and same-named agent specs stay distinct seats (`name`, `name-2`, ...) since each runs a different command. Non-zero exit, timeout (900s), or empty output degrades the seat honestly, like any provider failure. The command is operator-supplied (the `--builder-cmd` trust level), never taken from a repo or diff. Example wrapper: `examples/agent-seat-claude.sh`. Gate record: review-v7 BLOCKED (shell-exec major + same-name seat collapse, both real) → fixed → verification. Suite: 69 tests (13 new).
|
|
2246
|
+
|
|
2247
|
+
## 0.6.0 — 2026-11-03
|
|
2248
|
+
- **Distribution name: `codereview-ai`** (owner decision, 2026-11-03). The PyPI names `codereview` and `codereview-cli` are both taken by unrelated projects; `codereview-ai` was verified free and is now the package's distribution name. The installed command remains `codereview`. Publishing itself waits on the GitHub reinstatement, per the internal-first plan.
|
|
2249
|
+
- **Intake mode** (`codereview intake <issue-report.md> [--context FILE ...] [--repo DIR]`): the front half of the workflow. Until now every entry point started from a code change that already existed; intake starts from the problem. Solver models read the issue report (plus optional context files and the repo's file listing) and each produce a solution plan in a fixed, executable shape: diagnosis steps in order, ranked hypotheses each with the test that settles it, a numbered fix plan, prevention, risks and rollback, and closure criteria. The first configured solver that answers becomes `solution.md`; other solvers' plans are kept as alternates, and `intake.json` records the report's sha256, per-solver outcomes, and every truncation. Exit 0 = a plan was produced, 2 = every solver failed.
|
|
2250
|
+
- **Solver seats** (`providers.build_solvers`): the same presets, custom specs, and account pools as the reviewers, but answering in plain text — a solver-persona system prompt and, for Gemini, no JSON response mode (the reviewer callables are byte-for-byte unchanged in behaviour).
|
|
2251
|
+
- Intake never writes code and never gates: the builder implements `solution.md` and the existing review loop gates the change. Report and context pass through the same secret redaction as diffs and are size-capped, with truncations disclosed inside the prompt itself. Suite: 56 tests (12 new). Self-review: the intake diff went through the loop — round 1 BLOCKED (6 findings: a per-solver timeout and a worker cap were added, the redaction test's fixture was rebuilt from fragments, and two findings were rebutted with code evidence locked in by new wire-level tests), verification round 2 PASS with all prior blockers cleared by reviewer consensus.
|
|
2252
|
+
|
|
2253
|
+
## 0.5.0 — 2026-11-03
|
|
2254
|
+
- **Renamed: ReviewLoop is now CodeReview.** Package `codereview`, CLI `codereview`, builder env contract `CODEREVIEW_*`, GitHub commit-status context `codereview`, approval commits authored as CodeReview. Backwards compatibility is kept for one transition: a `reviewloop` import shim and CLI alias ship in the package, builders receive both `CODEREVIEW_*` and legacy `REVIEWLOOP_*` variables, and `$REVIEWLOOP_PROVIDERS` is still honoured. No behaviour changes. Note: the PyPI distribution name `codereview` is already taken by an unrelated project (a desktop diff viewer), so a future PyPI release will need a distinct distribution name; the GitHub repository name is unaffected.
|
|
2255
|
+
|
|
2256
|
+
## 0.4.2 — 2026-11-03
|
|
2257
|
+
External Review 3 — verdict was Approve; its one Minor is fixed:
|
|
2258
|
+
- **Approval commits are assembled in a temporary Git index.** The engine now builds the commit with `GIT_INDEX_FILE` pointed at a throwaway index (read-tree the step base → add the working tree → verify the staged diff is byte-identical to the approved diff → write-tree → commit-tree → HEAD moved by an update-ref guarded on the base). The v0.4.1 recovery path ran `git reset -q -- .` on a staged-diff mismatch, which reset the user's whole index and destroyed pre-staged state belonging to a resumed step. Now a refusal touches nothing: the temporary index is discarded and the user's index is exactly as it was. On success the real index is synced to the new HEAD — the state a normal commit leaves behind.
|
|
2259
|
+
- Regression test: a resumed step with pre-staged work that hits a forced staged mismatch keeps its staged entries after the refusal (`test_mismatch_refusal_preserves_staged_index`); the exact-commit test now also asserts a clean post-commit index. Suite: 44 tests. Full response: `review-3-response-2026-11-03.md`.
|
|
2260
|
+
|
|
2261
|
+
## 0.4.1 — 2026-11-03
|
|
2262
|
+
External Review 2 — one Major, verified real by reproduction, fixed:
|
|
2263
|
+
- **Workflow steps are isolated from the user's working tree.** A step starting fresh now requires a clean tree (staged, unstaged, and untracked changes all count; the workflow output directory is excepted): the engine stops with `needs_attention` and a message naming the offending paths instead of reviewing — and later committing — unrelated local edits as step work. A resumed step (already started, e.g. `changes_requested`) keeps its own in-progress tree. The output directory is also excluded from the reviewed diff and from intent-to-add, so the engine never reviews its own reports.
|
|
2264
|
+
- **Approval commits are exact.** On PASS the engine re-reads the diff and refuses to commit when its sha256 differs from the final reviewed report's; it then stages (output dir excluded) and verifies the staged diff is byte-identical to the approved diff before committing. Any mismatch resets the index and stops the step as `needs_attention` — an approval commit can no longer contain anything the reviewers did not approve.
|
|
2265
|
+
- Regression tests: dirty-tree refusal (agent never runs, HEAD unchanged, edits untouched), exact approval-commit contents, resumed-step behaviour, post-review tree mutation refusal. Suite: 43 tests. Full response: `review-2-response-2026-11-03.md`.
|
|
2266
|
+
|
|
2267
|
+
## 0.4.0 — 2026-11-03
|
|
2268
|
+
- **GitHub PR integration** (`reviewloop pr --repo OWNER/REPO --number N`, or `--pr <url>`): fetches the PR's metadata and unified diff from the GitHub API and reviews the diff where the work lives. Read path needs no token on public repos. `--post` (needs `GITHUB_TOKEN`) posts the report as a PR review comment — COMMENT by default, REQUEST_CHANGES on a blocked gate only with `--request-changes`; the bot never auto-approves — and sets a `reviewloop` commit status on the PR head (success / failure / pending from the gate). `--verify-previous` runs the verification round against the PR's current diff, so the review → fix → verify loop works across pushes. Tokens come from the environment only, are never logged, and authenticated requests refuse redirects.
|
|
2269
|
+
- Live proof: karpathy/llm-council PR #226 fetched and reviewed read-only with the real panel (gate PASS, degraded honestly labelled while one provider was quota-limited). The write path is covered by fake-transport unit tests; the first live post waits on our GitHub account reinstatement — that is the only account-gated piece.
|
|
2270
|
+
- Self-review of this release: 3 majors raised, all three rebutted with code evidence in the verification round (the checks/timeout/no-retry-loop facts were already in the code), 1 minor fixed (table-cell newline collapsing in PR comments). 39 unit tests.
|
|
2271
|
+
|
|
2272
|
+
## 0.3.0 — 2026-11-03
|
|
2273
|
+
The workflow spec's real gaps, closed:
|
|
2274
|
+
- **Account pools with visible fallback rotation.** Repeating a reviewer seat (`--provider gemini --provider gemini`, or custom specs sharing a name) builds a pool: accounts are tried in order (preset accounts read `GEMINI_API_KEY`, `GEMINI_API_KEY_2`, …), a quota/rate-limit/5xx/dead-key error rotates to the next account, and every report records which account served and each rotation (from → to → error). Never a silent substitution. Live proof: a dead first account rotated to the second, recorded in the report.
|
|
2275
|
+
- **Multi-step workflow engine** (`reviewloop workflow --file wf.json`): ordered steps with the spec's states (queued / running / changes_requested / verifying / approved / needs_attention), each step gated by the full review loop; on PASS the step's change is committed as its approval record and the workflow advances. State persists in `workflow-state.json` — rerunning resumes (approved steps are skipped); a `STOP` file pauses between steps; owner waivers are read from the state's `waivers` map.
|
|
2276
|
+
- **Owner waivers in the loop** (`--waive <finding-id>` or workflow-state waivers): waived blockers are recorded as `waived_by_owner` in the finding lifecycle and never sent to the builder.
|
|
2277
|
+
- **Round-to-round change summary:** each loop round records its file list and the per-file line-count delta vs the previous round.
|
|
2278
|
+
- Fixes found by the workflow tests and live runs: patches now include untracked files (intent-to-add) — a review blind to the agent's new files was the engine's worst failure mode; approval commits exclude the workflow's own output directory; JSON recovery hardened (repair retry + one fresh sample; prompts now ban double quotes inside string values, the main malformed-JSON cause).
|
|
2279
|
+
|
|
2280
|
+
## 0.2.1 — 2026-11-03
|
|
2281
|
+
From the reviewer↔coding-agent workflow spec (same day):
|
|
2282
|
+
- **Structured work-report contract enforced in the loop:** the builder packet now requires one disposition line per finding (`Finding <id>: fixed | rebutted | unresolved — evidence`); reports missing dispositions are flagged to the verification reviewers and recorded as `work_report_gaps` (they never clear a finding by themselves).
|
|
2283
|
+
- **Finding lifecycle in `loop-result.json`:** every blocking finding ends as `fixed_verified`, `rebutted_accepted`, or `unresolved`, plus a `workflow_state` (approved / changes_requested / needs_attention) matching the spec's step states.
|
|
2284
|
+
|
|
2285
|
+
## 0.2.0 — 2026-11-03
|
|
2286
|
+
- **Loop driver** (`reviewloop loop`): the full workflow as one command — reviewers review the working-tree diff; on BLOCKED, the findings packet is handed to a builder command (the coding agent: it receives `$REVIEWLOOP_PACKET`, edits the tree, writes its work report to `$REVIEWLOOP_RESPONSE`); a verification round then decides whether the workflow advances. Repeats up to `--max-rounds` builder rounds, then stops for a human. Writes per-round reports plus `loop-result.json` (`advanced: true` on PASS). Inconclusive rounds stop the loop — reviewer-infrastructure problems are never the builder's fault.
|
|
2287
|
+
- Live proof (internal panel): planted alert-flood change BLOCKED on round 1, scripted builder fixed it with a dedup guard + work report, verification PASS on round 2, loop advanced.
|
|
2288
|
+
|
|
2289
|
+
## 0.1.1 — 2026-11-03
|
|
2290
|
+
External review round 1 — both findings accepted and fixed:
|
|
2291
|
+
- Verification resolutions are now validated: the status must be in the allowed set, and a `fixed`/`invalid` claim needs non-empty evidence that shares real vocabulary with the diff, test output, prior finding, or builder response. Unsupported assertions are downgraded to `unclear` (block stays) and marked in the report.
|
|
2292
|
+
- `--strict` verification rounds now return INCONCLUSIVE when a reviewer failed (degraded), matching the documented strict behaviour of review rounds.
|
|
2293
|
+
- Regression tests for both findings; suite is 18 tests.
|
|
2294
|
+
|
|
2295
|
+
## 0.1.0 — 2026-11-03
|
|
2296
|
+
- Initial build: review → gate → verify loop.
|
|
2297
|
+
- Deterministic scan (secrets, injection, data-loss, alert-guard, disabled tests, debug leftovers) + project test commands as blockers.
|
|
2298
|
+
- Parallel provider review with correctness and operations lenses; consensus merge.
|
|
2299
|
+
- Verification round: prior blockers clear only on unanimous reviewer agreement.
|
|
2300
|
+
- Providers: Gemini, OpenAI, xAI, NVIDIA, DeepSeek presets + custom OpenAI-compatible specs.
|
|
2301
|
+
- Secret redaction before provider calls (and in stored report evidence); diff treated as untrusted data in prompts.
|
|
2302
|
+
- Provider JSON repair retry (one re-ask with an explicit escaping instruction) before a malformed reply counts as a provider error.
|
|
2303
|
+
- Live-verified 2026-11-03 on the internal Gemini+NVIDIA panel: deliberately buggy fixture blocked with 7 blocking findings (hardcoded secret, SQL injection, unconditional deletion, alert flood); guarded fixture passed after one verification round.
|