refuta 0.0.2__tar.gz → 0.0.3__tar.gz

{refuta-0.0.2 → refuta-0.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: refuta
-Version: 0.0.2
+Version: 0.0.3
 Summary: Refuta — Refutation-driven Agentic Code Security Platform for DGX Spark
 Author: Refuta contributors
 Author-email: "DAPADA Inc." <contact@dapada.co.kr>
@@ -67,21 +67,24 @@ refuta/
 ├── expected.json           # Ground truth used only by the independent evaluator (`refuta eval`)
 ├── results/                # fixture / qwen / qwen-cached run outputs and metrics
 └── tests/
-    └── test_refuta.py      # 20 unit, integration, and security-boundary regression tests
+    ├── test_refuta.py      # Unit, integration, and security-boundary regression tests
+    └── test_local_diagnostics.py # Local-response failure and diagnostic privacy regressions
 ```
 
 ---
 
 ## 3. Benchmark and verification results
 
-On a real host vLLM (`nvidia/Qwen3.6-35B-A3B-NVFP4`, `:8000`), Refuta found **4/4 real vulnerabilities (including 1 `fixed_verified` through bwrap-isolated patch verification)**, **refuted 2/2 safe cases protected by upstream guards (`refuted`, 0 false positives)**, with **0 quote or line-number evidence errors**, **0 raw secret leaks**, and the **untrusted `AGENTS.md` quarantined (`quarantine_verified: true`)**. All **20 unit and regression tests pass** (`1.74s`).
+On a real host vLLM (`nvidia/Qwen3.6-35B-A3B-NVFP4`, `:8000`), the recorded run (`--samples 3`) found **4/4 real vulnerabilities (including 1 `fixed_verified` through bwrap-isolated patch verification)**, **refuted 2/2 safe cases protected by upstream guards (`refuted`, 0 false positives)**, with **0 quote or line-number evidence errors**, **0 raw secret leaks**, and the **untrusted `AGENTS.md` quarantined (`quarantine_verified: true`)**. At recording time, all **22 unit and regression tests passed** (`1.46s`).
 
 | Mode | Vulnerabilities found | Safe cases refuted | False positives | Evidence errors | Secrets exposed | Model calls | Time |
 | --- | --- | --- | --- | --- | --- | --- | --- |
 | Baseline scanners only (reference) | 4/4 | 0/2 | **2** | — | 0 | 0 | 0.005 s |
-| Canned responses (`fixture`) | 4/4 (`fixed_verified` 1) | 2/2 | **0** | 0 | 0 | 5 (stub) | 0.283 s |
-| Real Qwen3.6 audit (`local`) | 4/4 (`fixed_verified` 1) | 2/2 | **0** | 0 | 0 | 5 | 35.012 s |
-| Re-run from cached investigations (`qwen-cached`) | 4/4 (`fixed_verified` 1) | 2/2 | **0** | 0 | 0 | 0 (5 cache hits) | 0.273 s |
+| Canned responses (`fixture`) | 4/4 (`fixed_verified` 1) | 2/2 | **0** | 0 | 0 | 5 (stub) | 0.354 s |
+| Real Qwen3.6 audit (`local`, `--samples 3`) | 4/4 (`fixed_verified` 1) | 2/2 | **0** | 0 | 0 | 15 | 114.173 s |
+| Re-run from cached investigations (`qwen-cached`) | 4/4 (`fixed_verified` 1) | 2/2 | **0** | 0 | 0 | 0 (15 cache hits) | 0.296 s |
+
+The `local` row is one recorded run. vLLM output is not bit-exact at temperature 0, so a single investigation can flip: on 2026-10-26, single-sample runs (`--samples 1`) matched the ground truth in 6 of 8 runs, and both failures left `CASE-02-GUARDED-ADMIN` `confirmed` instead of `refuted` (1 false positive). With `--samples 3` (strict-majority vote over independently adjudicated investigations), 10 of 11 runs matched; the other had no majority on CASE-02 and escalated it to `needs_review`. No run in either mode missed a vulnerability, produced an evidence error, or exposed a secret.
 
 - Real Qwen run: [report](https://github.com/memtomem/refuta/blob/main/results/qwen/report.md), [structured result](https://github.com/memtomem/refuta/blob/main/results/qwen/result.json), [SARIF](https://github.com/memtomem/refuta/blob/main/results/qwen/sarif.json), [metrics](https://github.com/memtomem/refuta/blob/main/results/qwen/metrics.json)
 - Cached re-run: [report](https://github.com/memtomem/refuta/blob/main/results/qwen-cached/report.md) and [metrics](https://github.com/memtomem/refuta/blob/main/results/qwen-cached/metrics.json)
@@ -94,8 +97,8 @@ On a real host vLLM (`nvidia/Qwen3.6-35B-A3B-NVFP4`, `:8000`), Refuta found **4/
 Run from a source checkout (`uv sync --group dev` creates `.venv`). After `pip install refuta`, use `refuta` instead of `.venv/bin/refuta`.
 
 ```bash
-# 1) All unit and integration tests (20)
-.venv/bin/pytest -q tests/test_refuta.py
+# 1) All unit and integration tests
+.venv/bin/pytest -q
 
 # 2) Static, secret, and authorization scanners only
 .venv/bin/refuta scan \
@@ -111,6 +114,7 @@ Run from a source checkout (`uv sync --group dev` creates `.venv`). After `pip i
 .venv/bin/refuta audit \
   --repo sample/repo \
   --runner local \
+  --samples 3 \
   --patches patches/manifest.json \
   --out results/qwen
 
@@ -123,11 +127,23 @@ Run from a source checkout (`uv sync --group dev` creates `.venv`). After `pip i
 # Offline, no model needed: --runner fixture --fixture fixtures/responses.json
 ```
 
-`audit` exits 0 when the run completes and 2 when some candidates failed (partial); `eval` exits 1 when the result does not match the ground truth.
+`audit` defaults to 3 independent investigations per candidate for `--runner local`, with a strict-majority vote; other runners default to 1. Set `--samples N` to override either default (`N >= 1`). On an uncached audit, the local default uses about three times as many model calls as `--samples 1`. Existing sample-0 cache entries are reused; only missing samples require new calls.
+
+Local investigations run sequentially, and remaining samples are still attempted after a failure. The HTTP timeout (`RepoPolicy.max_timeout_sec`, default 120 seconds) applies to each request rather than setting a deadline for the entire audit. A stalled endpoint can therefore cause three consecutive timeout waits per candidate with the local default. Use `--samples 1` to reduce the number of requests when troubleshooting the endpoint.
+
+`audit` exits 0 when the run completes and 2 when any investigation sample fails (`partial`), even if the other samples reach a strict majority and decide the finding. Check `result.json.errors` for failed samples and each finding's `sample_statuses` for the vote. `eval` exits 1 when the result does not match the ground truth.
+
+### Local-response diagnostics
+
+When a local model response has invalid UTF-8, malformed JSON, an invalid response envelope, or investigation content that is not a JSON object, the audit automatically saves a diagnostic under `<out>/diagnostics/<run_id>/<candidate_id>-sample<N>.json`. The sample number is zero-based. Each entry in `result.json.errors` points to its diagnostic; the same error list is saved in the audit's SQLite record. Failed responses are not cached. Successful responses create no diagnostic files, and a diagnostic write failure is recorded while the audit continues as `partial`.
+
+Diagnostics contain the model, failure stage, response text, completion reason and token usage when available, and candidate/rule/sample identity. JSON errors also include the exact parser input after redaction, with character position and one-based line/column coordinates. Secrets in that parser input use equal-length `*` masks so the coordinates remain accurate after code-fence or reasoning-block removal and fallback extraction. Other diagnostic text uses the existing hashed secret markers, including for tokens split by Unicode escapes inside nested JSON or following short escapes such as `\n`. Diagnostic masking conservatively recognizes token prefixes within surrounding text and extends masking beyond scanner length caps; it can redact more text than the source scanner. The serialized diagnostic is checked again before writing.
+
+Diagnostic directories are restricted to the owner (`0700`), files use `0600`, and symlink directories are rejected. Diagnostics are ignored by Git. Redaction covers the existing GitHub-token, `sk-`, and AWS-access-key families; truncated tokens or tokens interrupted by invalid encoding may not match. A field with unresolved escape obfuscation is replaced entirely with a redaction marker; parser input instead uses stars while preserving its length and line breaks. Diagnostics can still contain source code or other sensitive values and should be handled as sensitively as the audited snapshot. These permissions apply to diagnostics, not to the audit's other output files. HTTP errors and timeouts retain their existing error handling without response-body capture.
 
 ---
 
-## 5. Security and verification boundaries covered by tests (`tests/test_refuta.py`)
+## 5. Security and verification boundaries covered by tests
 
 1. **Untrusted instruction-file quarantine and credential stripping (`RepoProvider`)**: when a snapshot is created, tokens in `.git/config` are removed and `AGENTS.md`, `CLAUDE.md`, `.claude/`, `.codex/`, and `.github/copilot-instructions.md` are quarantined automatically.
 2. **Loop 1 diff scope (`compute_diff_scope`)**: computes exactly which files were added, modified, or deleted between two snapshots.
@@ -136,19 +152,21 @@ Run from a source checkout (`uv sync --group dev` creates `.venv`). After `pip i
 5. **Deterministic evidence verifier (`verifier.py`)**: checks that the `vulnerable_quotes` and `guard_quotes` the model cites exist verbatim in the original files, and computes line numbers itself. Missing files, path escapes (`../`), and altered code quotes are rejected immediately as `invalid_evidence`.
 6. **Strict refutation**: even if the model claims `is_refuted: true`, the refutation is rejected (`needs_review`) unless it presents guard-code quotes (`guard_quotes`) verified against the snapshot files.
 7. **Option D cross-model review (`adjudicate_cross_review`)**: adjudicates agreement and disagreement between the primary analyst and a second refuting reviewer, switching to `refuted` only when the refuting side presents a valid guard-code quote.
-8. **`bwrap` (bubblewrap) isolated patch and scanner runner (`sandbox.py`)**:
+8. **Sample voting (`adjudicate_samples`, `audit --samples N`)**: runs N independent investigations per candidate, adjudicates each one separately, and keeps a status only when a strict majority reaches it; anything else escalates to `needs_review`. A sample counts as `refuted` only if its guard quote verified, so a single-sample refutation without evidence cannot win the vote. Per-sample statuses are kept in `sample_statuses`, and each sample has its own cache entry.
+9. **`bwrap` (bubblewrap) isolated patch and scanner runner (`sandbox.py`)**:
    - Isolates test and scanner processes with `/usr/bin/bwrap` (`--unshare-all --die-with-parent --new-session --ro-bind / / --tmpfs $HOME --ro-bind <venv> <venv> --ro-bind <base interpreter under $HOME, e.g. uv Python> --tmpfs /tmp --ro-bind <work_repo> /tmp/work --clearenv`).
    - Demonstrated inside the sandbox: (1) `$HOME` is not accessible, (2) the source under `/tmp/work` cannot be written (`Read-only file system`), (3) neither the host's `127.0.0.1:8000` nor external networks are reachable (`Network is unreachable`).
    - A finding is promoted to `fixed_verified` only when all four conditions hold: (1) test files (`tests/`) are unmodified, (2) the security test fails before the patch (the vulnerability reproduces), (3) the security test passes after the patch, (4) the functional regression tests pass after the patch.
    - Execution is blocked when an external binary or model weight does not match its pinned SHA256 digest (`verify_artifact_sha256`).
-9. **Default deny for sending code off-host, and a memory guard (`RunnerRouter`)**: with `local_only=True` (the default), `ClaudeRunner` and external `CodexRunner` calls are blocked; local inference is refused with `InsufficientMemoryError` when `MemAvailable` is below the threshold (`min_mem_available_gb`, default 8).
-10. **Accepted-risk expiry and invalidation on file change (`RefutaStore`)**: an `accepted_risk` exception is released and the finding restored to `confirmed` when the exception expires or the SHA256 of the target file changes.
-11. **Continuous security loops and publishing safeguards (`loops.py`, `publisher.py`)**:
+10. **Default deny for sending code off-host, and a memory guard (`RunnerRouter`)**: with `local_only=True` (the default), `ClaudeRunner` and external `CodexRunner` calls are blocked; local inference is refused with `InsufficientMemoryError` when `MemAvailable` is below the threshold (`min_mem_available_gb`, default 8).
+11. **Accepted-risk expiry and invalidation on file change (`RefutaStore`)**: an `accepted_risk` exception is released and the finding restored to `confirmed` when the exception expires or the SHA256 of the target file changes.
+12. **Continuous security loops and publishing safeguards (`loops.py`, `publisher.py`)**:
     - **Loop 3 (new-CVE response)**: queries the deployment/SBOM graph to identify affected services, environments, and runtime images immediately, without any code change.
     - **Loop 4 (attack-surface reduction)**: automatically identifies unauthenticated public routes, debug endpoints, and privileged admin paths.
     - **Loop 6 (post-deploy re-verification)**: fails while a pre-patch commit is still running in production, and completes only once the patched commit is deployed and passes its health check.
     - **Quiet-window guard (`QuietWindow`, `is_in_quiet_window`)**: tells heavy scheduled audits when to yield the host, e.g. to another model server sharing the GPU. Windows are configured by the caller (time zone, start/end, weekdays); the test uses a DGX Spark schedule (Mon/Thu 03:50–09:00 KST, daily 06:50–07:30 KST) as its example.
     - **Publishing safeguards**: GitHub webhook HMAC-SHA256 signature verification, blocking publication on a commit SHA mismatch (`StaleCommitPublishError`), and automatic redaction of secret tokens in outbound text.
+13. **Local-response diagnostics (`tests/test_local_diagnostics.py`)**: redacts supported tokens (including overlapping patterns and Unicode escapes), preserves parser coordinates, separates runs, restricts file permissions, rejects symlink directories, and keeps partial audits usable when diagnostic writes fail. Non-object local responses are rejected before caching, and malformed legacy cache entries are re-investigated.
 
 ---
 

{refuta-0.0.2 → refuta-0.0.3}/README.md

@@ -41,21 +41,24 @@ refuta/
 ├── expected.json           # Ground truth used only by the independent evaluator (`refuta eval`)
 ├── results/                # fixture / qwen / qwen-cached run outputs and metrics
 └── tests/
-    └── test_refuta.py      # 20 unit, integration, and security-boundary regression tests
+    ├── test_refuta.py      # Unit, integration, and security-boundary regression tests
+    └── test_local_diagnostics.py # Local-response failure and diagnostic privacy regressions
 ```
 
 ---
 
 ## 3. Benchmark and verification results
 
-On a real host vLLM (`nvidia/Qwen3.6-35B-A3B-NVFP4`, `:8000`), Refuta found **4/4 real vulnerabilities (including 1 `fixed_verified` through bwrap-isolated patch verification)**, **refuted 2/2 safe cases protected by upstream guards (`refuted`, 0 false positives)**, with **0 quote or line-number evidence errors**, **0 raw secret leaks**, and the **untrusted `AGENTS.md` quarantined (`quarantine_verified: true`)**. All **20 unit and regression tests pass** (`1.74s`).
+On a real host vLLM (`nvidia/Qwen3.6-35B-A3B-NVFP4`, `:8000`), the recorded run (`--samples 3`) found **4/4 real vulnerabilities (including 1 `fixed_verified` through bwrap-isolated patch verification)**, **refuted 2/2 safe cases protected by upstream guards (`refuted`, 0 false positives)**, with **0 quote or line-number evidence errors**, **0 raw secret leaks**, and the **untrusted `AGENTS.md` quarantined (`quarantine_verified: true`)**. At recording time, all **22 unit and regression tests passed** (`1.46s`).
 
 | Mode | Vulnerabilities found | Safe cases refuted | False positives | Evidence errors | Secrets exposed | Model calls | Time |
 | --- | --- | --- | --- | --- | --- | --- | --- |
 | Baseline scanners only (reference) | 4/4 | 0/2 | **2** | — | 0 | 0 | 0.005 s |
-| Canned responses (`fixture`) | 4/4 (`fixed_verified` 1) | 2/2 | **0** | 0 | 0 | 5 (stub) | 0.283 s |
-| Real Qwen3.6 audit (`local`) | 4/4 (`fixed_verified` 1) | 2/2 | **0** | 0 | 0 | 5 | 35.012 s |
-| Re-run from cached investigations (`qwen-cached`) | 4/4 (`fixed_verified` 1) | 2/2 | **0** | 0 | 0 | 0 (5 cache hits) | 0.273 s |
+| Canned responses (`fixture`) | 4/4 (`fixed_verified` 1) | 2/2 | **0** | 0 | 0 | 5 (stub) | 0.354 s |
+| Real Qwen3.6 audit (`local`, `--samples 3`) | 4/4 (`fixed_verified` 1) | 2/2 | **0** | 0 | 0 | 15 | 114.173 s |
+| Re-run from cached investigations (`qwen-cached`) | 4/4 (`fixed_verified` 1) | 2/2 | **0** | 0 | 0 | 0 (15 cache hits) | 0.296 s |
+
+The `local` row is one recorded run. vLLM output is not bit-exact at temperature 0, so a single investigation can flip: on 2026-10-26, single-sample runs (`--samples 1`) matched the ground truth in 6 of 8 runs, and both failures left `CASE-02-GUARDED-ADMIN` `confirmed` instead of `refuted` (1 false positive). With `--samples 3` (strict-majority vote over independently adjudicated investigations), 10 of 11 runs matched; the other had no majority on CASE-02 and escalated it to `needs_review`. No run in either mode missed a vulnerability, produced an evidence error, or exposed a secret.
 
 - Real Qwen run: [report](https://github.com/memtomem/refuta/blob/main/results/qwen/report.md), [structured result](https://github.com/memtomem/refuta/blob/main/results/qwen/result.json), [SARIF](https://github.com/memtomem/refuta/blob/main/results/qwen/sarif.json), [metrics](https://github.com/memtomem/refuta/blob/main/results/qwen/metrics.json)
 - Cached re-run: [report](https://github.com/memtomem/refuta/blob/main/results/qwen-cached/report.md) and [metrics](https://github.com/memtomem/refuta/blob/main/results/qwen-cached/metrics.json)
@@ -68,8 +71,8 @@ On a real host vLLM (`nvidia/Qwen3.6-35B-A3B-NVFP4`, `:8000`), Refuta found **4/
 Run from a source checkout (`uv sync --group dev` creates `.venv`). After `pip install refuta`, use `refuta` instead of `.venv/bin/refuta`.
 
 ```bash
-# 1) All unit and integration tests (20)
-.venv/bin/pytest -q tests/test_refuta.py
+# 1) All unit and integration tests
+.venv/bin/pytest -q
 
 # 2) Static, secret, and authorization scanners only
 .venv/bin/refuta scan \
@@ -85,6 +88,7 @@ Run from a source checkout (`uv sync --group dev` creates `.venv`). After `pip i
 .venv/bin/refuta audit \
   --repo sample/repo \
   --runner local \
+  --samples 3 \
   --patches patches/manifest.json \
   --out results/qwen
 
@@ -97,11 +101,23 @@ Run from a source checkout (`uv sync --group dev` creates `.venv`). After `pip i
 # Offline, no model needed: --runner fixture --fixture fixtures/responses.json
 ```
 
-`audit` exits 0 when the run completes and 2 when some candidates failed (partial); `eval` exits 1 when the result does not match the ground truth.
+`audit` defaults to 3 independent investigations per candidate for `--runner local`, with a strict-majority vote; other runners default to 1. Set `--samples N` to override either default (`N >= 1`). On an uncached audit, the local default uses about three times as many model calls as `--samples 1`. Existing sample-0 cache entries are reused; only missing samples require new calls.
+
+Local investigations run sequentially, and remaining samples are still attempted after a failure. The HTTP timeout (`RepoPolicy.max_timeout_sec`, default 120 seconds) applies to each request rather than setting a deadline for the entire audit. A stalled endpoint can therefore cause three consecutive timeout waits per candidate with the local default. Use `--samples 1` to reduce the number of requests when troubleshooting the endpoint.
+
+`audit` exits 0 when the run completes and 2 when any investigation sample fails (`partial`), even if the other samples reach a strict majority and decide the finding. Check `result.json.errors` for failed samples and each finding's `sample_statuses` for the vote. `eval` exits 1 when the result does not match the ground truth.
+
+### Local-response diagnostics
+
+When a local model response has invalid UTF-8, malformed JSON, an invalid response envelope, or investigation content that is not a JSON object, the audit automatically saves a diagnostic under `<out>/diagnostics/<run_id>/<candidate_id>-sample<N>.json`. The sample number is zero-based. Each entry in `result.json.errors` points to its diagnostic; the same error list is saved in the audit's SQLite record. Failed responses are not cached. Successful responses create no diagnostic files, and a diagnostic write failure is recorded while the audit continues as `partial`.
+
+Diagnostics contain the model, failure stage, response text, completion reason and token usage when available, and candidate/rule/sample identity. JSON errors also include the exact parser input after redaction, with character position and one-based line/column coordinates. Secrets in that parser input use equal-length `*` masks so the coordinates remain accurate after code-fence or reasoning-block removal and fallback extraction. Other diagnostic text uses the existing hashed secret markers, including for tokens split by Unicode escapes inside nested JSON or following short escapes such as `\n`. Diagnostic masking conservatively recognizes token prefixes within surrounding text and extends masking beyond scanner length caps; it can redact more text than the source scanner. The serialized diagnostic is checked again before writing.
+
+Diagnostic directories are restricted to the owner (`0700`), files use `0600`, and symlink directories are rejected. Diagnostics are ignored by Git. Redaction covers the existing GitHub-token, `sk-`, and AWS-access-key families; truncated tokens or tokens interrupted by invalid encoding may not match. A field with unresolved escape obfuscation is replaced entirely with a redaction marker; parser input instead uses stars while preserving its length and line breaks. Diagnostics can still contain source code or other sensitive values and should be handled as sensitively as the audited snapshot. These permissions apply to diagnostics, not to the audit's other output files. HTTP errors and timeouts retain their existing error handling without response-body capture.
 
 ---
 
-## 5. Security and verification boundaries covered by tests (`tests/test_refuta.py`)
+## 5. Security and verification boundaries covered by tests
 
 1. **Untrusted instruction-file quarantine and credential stripping (`RepoProvider`)**: when a snapshot is created, tokens in `.git/config` are removed and `AGENTS.md`, `CLAUDE.md`, `.claude/`, `.codex/`, and `.github/copilot-instructions.md` are quarantined automatically.
 2. **Loop 1 diff scope (`compute_diff_scope`)**: computes exactly which files were added, modified, or deleted between two snapshots.
@@ -110,19 +126,21 @@ Run from a source checkout (`uv sync --group dev` creates `.venv`). After `pip i
 5. **Deterministic evidence verifier (`verifier.py`)**: checks that the `vulnerable_quotes` and `guard_quotes` the model cites exist verbatim in the original files, and computes line numbers itself. Missing files, path escapes (`../`), and altered code quotes are rejected immediately as `invalid_evidence`.
 6. **Strict refutation**: even if the model claims `is_refuted: true`, the refutation is rejected (`needs_review`) unless it presents guard-code quotes (`guard_quotes`) verified against the snapshot files.
 7. **Option D cross-model review (`adjudicate_cross_review`)**: adjudicates agreement and disagreement between the primary analyst and a second refuting reviewer, switching to `refuted` only when the refuting side presents a valid guard-code quote.
-8. **`bwrap` (bubblewrap) isolated patch and scanner runner (`sandbox.py`)**:
+8. **Sample voting (`adjudicate_samples`, `audit --samples N`)**: runs N independent investigations per candidate, adjudicates each one separately, and keeps a status only when a strict majority reaches it; anything else escalates to `needs_review`. A sample counts as `refuted` only if its guard quote verified, so a single-sample refutation without evidence cannot win the vote. Per-sample statuses are kept in `sample_statuses`, and each sample has its own cache entry.
+9. **`bwrap` (bubblewrap) isolated patch and scanner runner (`sandbox.py`)**:
    - Isolates test and scanner processes with `/usr/bin/bwrap` (`--unshare-all --die-with-parent --new-session --ro-bind / / --tmpfs $HOME --ro-bind <venv> <venv> --ro-bind <base interpreter under $HOME, e.g. uv Python> --tmpfs /tmp --ro-bind <work_repo> /tmp/work --clearenv`).
    - Demonstrated inside the sandbox: (1) `$HOME` is not accessible, (2) the source under `/tmp/work` cannot be written (`Read-only file system`), (3) neither the host's `127.0.0.1:8000` nor external networks are reachable (`Network is unreachable`).
    - A finding is promoted to `fixed_verified` only when all four conditions hold: (1) test files (`tests/`) are unmodified, (2) the security test fails before the patch (the vulnerability reproduces), (3) the security test passes after the patch, (4) the functional regression tests pass after the patch.
    - Execution is blocked when an external binary or model weight does not match its pinned SHA256 digest (`verify_artifact_sha256`).
-9. **Default deny for sending code off-host, and a memory guard (`RunnerRouter`)**: with `local_only=True` (the default), `ClaudeRunner` and external `CodexRunner` calls are blocked; local inference is refused with `InsufficientMemoryError` when `MemAvailable` is below the threshold (`min_mem_available_gb`, default 8).
-10. **Accepted-risk expiry and invalidation on file change (`RefutaStore`)**: an `accepted_risk` exception is released and the finding restored to `confirmed` when the exception expires or the SHA256 of the target file changes.
-11. **Continuous security loops and publishing safeguards (`loops.py`, `publisher.py`)**:
+10. **Default deny for sending code off-host, and a memory guard (`RunnerRouter`)**: with `local_only=True` (the default), `ClaudeRunner` and external `CodexRunner` calls are blocked; local inference is refused with `InsufficientMemoryError` when `MemAvailable` is below the threshold (`min_mem_available_gb`, default 8).
+11. **Accepted-risk expiry and invalidation on file change (`RefutaStore`)**: an `accepted_risk` exception is released and the finding restored to `confirmed` when the exception expires or the SHA256 of the target file changes.
+12. **Continuous security loops and publishing safeguards (`loops.py`, `publisher.py`)**:
     - **Loop 3 (new-CVE response)**: queries the deployment/SBOM graph to identify affected services, environments, and runtime images immediately, without any code change.
     - **Loop 4 (attack-surface reduction)**: automatically identifies unauthenticated public routes, debug endpoints, and privileged admin paths.
     - **Loop 6 (post-deploy re-verification)**: fails while a pre-patch commit is still running in production, and completes only once the patched commit is deployed and passes its health check.
     - **Quiet-window guard (`QuietWindow`, `is_in_quiet_window`)**: tells heavy scheduled audits when to yield the host, e.g. to another model server sharing the GPU. Windows are configured by the caller (time zone, start/end, weekdays); the test uses a DGX Spark schedule (Mon/Thu 03:50–09:00 KST, daily 06:50–07:30 KST) as its example.
     - **Publishing safeguards**: GitHub webhook HMAC-SHA256 signature verification, blocking publication on a commit SHA mismatch (`StaleCommitPublishError`), and automatic redaction of secret tokens in outbound text.
+13. **Local-response diagnostics (`tests/test_local_diagnostics.py`)**: redacts supported tokens (including overlapping patterns and Unicode escapes), preserves parser coordinates, separates runs, restricts file permissions, rejects symlink directories, and keeps partial audits usable when diagnostic writes fail. Non-object local responses are rejected before caching, and malformed legacy cache entries are re-investigated.
 
 ---
 

{refuta-0.0.2 → refuta-0.0.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "refuta"
-version = "0.0.2"
+version = "0.0.3"
 description = "Refuta — Refutation-driven Agentic Code Security Platform for DGX Spark"
 readme = "README.md"
 authors = [

{refuta-0.0.2 → refuta-0.0.3}/refuta/__init__.py

@@ -1,3 +1,3 @@
 """Refuta — Refutation-driven Agentic Code Security Platform."""
 
-__version__ = "0.0.2"
+__version__ = "0.0.3"

{refuta-0.0.2 → refuta-0.0.3}/refuta/cli.py

@@ -3,10 +3,13 @@
 from __future__ import annotations
 
 import argparse
+from contextlib import ExitStack
 from datetime import datetime, timezone
 import hashlib
 import json
+import os
 from pathlib import Path
+import secrets
 import sys
 import time
 from typing import Any
@@ -19,12 +22,15 @@ from .runners import (
     CodexRunner,
     FixtureRunner,
     LocalRunner,
+    LocalResponseError,
+    PROMPT_VERSION,
     RunnerRouter,
+    mask_diagnostic_text,
 )
 from .scanners import SECRET_PATTERNS, scan_snapshot
 from .schemas import AuditRunResult, EvidenceQuote, Finding, FindingStatus, RepoPolicy
 from .store import RefutaStore
-from .verifier import adjudicate_finding, verify_patch_in_sandbox, verify_quote_in_snapshot
+from .verifier import adjudicate_finding, adjudicate_samples, verify_patch_in_sandbox, verify_quote_in_snapshot
 
 
 def render_markdown_report(result: AuditRunResult) -> str:
@@ -138,6 +144,55 @@ def render_sarif_report(result: AuditRunResult) -> dict[str, Any]:
     }
 
 
+def _write_local_response_diagnostic(
+    out_path: Path,
+    run_id: str,
+    candidate_id: str,
+    rule_id: str,
+    sample_index: int,
+    error: LocalResponseError,
+) -> str:
+    """Write a private diagnostic through directory descriptors without following symlinks."""
+    payload = {
+        **error.diagnostic,
+        "run_id": run_id,
+        "candidate_id": candidate_id,
+        "rule_id": rule_id,
+        "sample_index": sample_index,
+    }
+    serialized = json.dumps(payload, indent=2)
+    # Validate the serialized file without changing the saved parse coordinates.
+    if mask_diagnostic_text(serialized) != serialized:
+        raise ValueError("Supported secret pattern remained in diagnostic")
+    diagnostic_root = out_path / "diagnostics"
+    diagnostic_root.mkdir(mode=0o700, exist_ok=True)
+    directory_flags = os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW
+    filename = f"{candidate_id}-sample{sample_index}.json"
+    with ExitStack() as stack:
+        root_fd = os.open(diagnostic_root, directory_flags)
+        stack.callback(os.close, root_fd)
+        os.fchmod(root_fd, 0o700)
+        try:
+            os.mkdir(run_id, mode=0o700, dir_fd=root_fd)
+        except FileExistsError:
+            pass
+        run_fd = os.open(run_id, directory_flags, dir_fd=root_fd)
+        stack.callback(os.close, run_fd)
+        os.fchmod(run_fd, 0o700)
+        fd = os.open(
+            filename, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW,
+            0o600, dir_fd=run_fd,
+        )
+        stack.callback(os.close, fd)
+        try:
+            with os.fdopen(fd, "w", encoding="utf-8", closefd=False) as diagnostic_file:
+                diagnostic_file.write(serialized)
+        except OSError:
+            os.unlink(filename, dir_fd=run_fd)
+            raise
+    return (Path("diagnostics") / run_id / filename).as_posix()
+
+
 def run_audit_pipeline(
     repo_path: str | Path,
     out_dir: str | Path,
@@ -147,8 +202,13 @@ def run_audit_pipeline(
     policy: RepoPolicy | None = None,
     patches_manifest_path: str | Path | None = None,
     only_files: set[str] | None = None,
+    samples: int | None = None,
 ) -> AuditRunResult:
-    """Execute the full Refuta pipeline: Snapshot -> Scan -> Investigate & Refute -> Verify -> Store."""
+    """Run the audit pipeline with 3 samples for local and 1 for other runners by default."""
+    if samples is None:
+        samples = 3 if runner_name == "local" else 1
+    if samples < 1:
+        raise ValueError("samples must be >= 1")
     provider = RepoProvider()
     resolved_policy = policy or RepoPolicy(repo_id=Path(repo_path).resolve().name)
     snapshot = provider.create_snapshot(repo_path, repo_id=resolved_policy.repo_id, read_only=True)
@@ -178,6 +238,12 @@ def run_audit_pipeline(
     findings: list[Finding] = []
     model_calls = 0
     cache_hits = 0
+    # Allocate identity before investigations so every diagnostic belongs to a run.
+    # The random suffix separates back-to-back audits within the same second.
+    run_id = (
+        f"run-{datetime.now(timezone.utc).strftime('%Y%m%dT%H%M%SZ')}"
+        f"-{snapshot.snapshot_hash[:8]}-{secrets.token_hex(3)}"
+    )
 
     try:
         candidates = scan_snapshot(snapshot.snapshot_dir, only_files=only_files)
@@ -192,40 +258,58 @@ def run_audit_pipeline(
                 rel: snapshot.file_hashes.get(rel, "")
                 for rel in [cand.file_path, *cand.related_files]
             }
-            cache_key = store.compute_cache_key(
-                rule_id=cand.rule_id,
-                file_path=cand.file_path,
-                related_hashes=related_hashes,
-                model=selected_runner.model_name,
-            )
-
-            cached_payload = store.get_cached_investigation(cache_key)
-            if cached_payload is not None:
-                agent_payload = cached_payload
-                cache_hits += 1
-            else:
-                try:
-                    agent_payload = selected_runner.investigate_and_refute(
-                        snapshot.snapshot_dir,
-                        cand,
-                        resolved_policy,
-                    )
-                    model_calls += 1
-                    digest = hashlib.sha256(
-                        json.dumps(related_hashes, sort_keys=True).encode("utf-8")
-                    ).hexdigest()
-                    store.put_cached_investigation(
-                        cache_key=cache_key,
-                        rule_id=cand.rule_id,
-                        file_hashes_digest=digest,
-                        model=selected_runner.model_name,
-                        payload=agent_payload,
-                    )
-                except Exception as exc:
-                    errors.append(f"{cand.candidate_id} ({cand.rule_id}): {exc}")
-                    agent_payload = None
+            payloads: list[dict[str, Any] | None] = []
+            for sample_index in range(samples):
+                cache_key = store.compute_cache_key(
+                    rule_id=cand.rule_id,
+                    file_path=cand.file_path,
+                    related_hashes=related_hashes,
+                    model=selected_runner.model_name,
+                    prompt_version=PROMPT_VERSION,
+                    sample_index=sample_index,
+                )
 
-            finding = adjudicate_finding(snapshot.snapshot_dir, cand, agent_payload)
+                cached_payload = store.get_cached_investigation(cache_key)
+                # Older local runs could cache non-object JSON before failing
+                # adjudication. Re-investigate those entries instead of replaying them.
+                if selected_runner.runner_name == "local" and not isinstance(cached_payload, dict):
+                    cached_payload = None
+                if cached_payload is not None:
+                    agent_payload = cached_payload
+                    cache_hits += 1
+                else:
+                    try:
+                        agent_payload = selected_runner.investigate_and_refute(
+                            snapshot.snapshot_dir,
+                            cand,
+                            resolved_policy,
+                        )
+                        model_calls += 1
+                        digest = hashlib.sha256(
+                            json.dumps(related_hashes, sort_keys=True).encode("utf-8")
+                        ).hexdigest()
+                        store.put_cached_investigation(
+                            cache_key=cache_key,
+                            rule_id=cand.rule_id,
+                            file_hashes_digest=digest,
+                            model=selected_runner.model_name,
+                            payload=agent_payload,
+                        )
+                    except Exception as exc:
+                        error_message = f"{cand.candidate_id} ({cand.rule_id}) sample {sample_index}: {exc}"
+                        if isinstance(exc, LocalResponseError):
+                            try:
+                                diagnostic_path = _write_local_response_diagnostic(
+                                    out_path, run_id, cand.candidate_id, cand.rule_id, sample_index, exc,
+                                )
+                                error_message += f"; diagnostic: {diagnostic_path}"
+                            except (OSError, ValueError) as diagnostic_exc:
+                                error_message += f"; diagnostic write failed ({type(diagnostic_exc).__name__})"
+                        errors.append(mask_diagnostic_text(error_message))
+                        agent_payload = None
+                payloads.append(agent_payload)
+
+            finding = adjudicate_samples(snapshot.snapshot_dir, cand, payloads)
             findings.append(finding)
 
         findings = store.apply_risk_exceptions(
@@ -256,8 +340,6 @@ def run_audit_pipeline(
 
         elapsed = time.perf_counter() - t0
         run_status = "completed" if not errors else "partial"
-        run_id = f"run-{datetime.now(timezone.utc).strftime('%Y%m%dT%H%M%SZ')}-{snapshot.snapshot_hash[:8]}"
-
         run_result = AuditRunResult(
             run_id=run_id,
             repo_id=resolved_policy.repo_id,
@@ -491,6 +573,12 @@ def main(argv: list[str] | None = None) -> int:
     p_audit.add_argument("--fixture", help="Path to fixture JSON when --runner fixture is used")
     p_audit.add_argument("--state", help="Path to SQLite state DB")
     p_audit.add_argument("--patches", help="Optional path to patches manifest JSON for isolated patch verification")
+    p_audit.add_argument(
+        "--samples",
+        type=int,
+        default=None,
+        help="Independent investigations per candidate; strict-majority vote (default: 3 for local, 1 otherwise)",
+    )
     p_audit.add_argument(
         "--allow-external",
         action="store_true",
@@ -558,6 +646,7 @@ def main(argv: list[str] | None = None) -> int:
             state_db_path=args.state,
             policy=policy,
             patches_manifest_path=args.patches,
+            samples=args.samples,
         )
         summary = {
             "run_id": res.run_id,