@tamsensedev/runner 0.13.0 → 0.19.0

package/library/skills/false-positive-check/references/deep-verification.md

@@ -1,113 +1,113 @@
-# Deep Verification
-
-Full verification for complex bugs. Use when routing in SKILL.md selects the deep path, or when standard verification escalates.
-
-## If Escalated from Standard
-
-1. Review all evidence gathered during the standard pass; do not repeat completed work.
-2. Identify which phases below are already satisfied by existing evidence.
-3. Continue from where standard verification left off.
-4. Keep and reference all prior findings.
-
-## Phases
-
-Work through the phases in order for each bug (Bug #N). Keep a checklist in your notes and mark a phase done only when you have concrete evidence for it. Sub-steps within a phase are independent; do them in any order, but never start a phase before the one it depends on is complete. If the bug path is large, you may hand a phase's data-flow tracing to the `code-explorer` agent, with the bug description and prior evidence in the prompt; you still judge the result.
-
-```
-Phase 1: Data flow analysis
-  1.1 Map trust boundaries and trace data flow
-  1.2 Research API contracts and safety guarantees
-  1.3 Environment protection analysis
-  1.4 Cross-reference analysis
-Phase 2: Exploitability verification (after phase 1)
-  2.1 Confirm attacker controls input data
-  2.2 Mathematical bounds verification (or N/A)
-  2.3 Race condition feasibility proof (or N/A)
-  2.4 Adversarial analysis (after 2.1 to 2.3)
-Phase 3: Impact assessment (after phase 2)
-  3.1 Demonstrate real security impact
-  3.2 Primary control vs defense in depth
-Phase 4: PoC creation (after phase 3)
-  4.1 Pseudocode PoC with data flow diagram
-  4.2 Executable PoC if feasible
-  4.3 Unit test PoC if feasible
-  4.4 Negative PoC: exploit preconditions
-  4.5 Verify the PoC demonstrates the vulnerability
-Phase 5: Devil's advocate review (after phase 4)
-Gate review (after phase 5)
-```
-
-Apply all 13 checklist items from [false-positive-patterns.md](false-positive-patterns.md) to each bug.
-
-## Phase Requirements
-
-The list above names every phase. Below are the key pitfalls and decision criteria for each: focus on what you might get wrong.
-
-### Phase 1: Data Flow Analysis
-
-**1.1**: Map trust boundaries (internal/trusted vs external/untrusted) and trace data from source to the alleged vulnerability. Apply class-specific verification from [bug-class-verification.md](bug-class-verification.md). **Key pitfall**: analyzing code in isolation without tracing the full validation chain. Conditional logic upstream may make the vulnerable code mathematically unreachable (see [false-positive-patterns.md](false-positive-patterns.md) items 1 and 1a).
-
-**1.2**: Check API contracts before claiming overflows or injections: many APIs have built-in protection (bounds checks, parameter binding, escaping) that prevents the alleged issue regardless of inputs.
-
-**1.3**: Before concluding vulnerability, verify that no compiler, runtime, OS, or framework protection prevents exploitation. Mitigations like ASLR and stack canaries raise the bar but do not eliminate the vulnerability itself. Distinguish "prevents exploitation entirely" (Rust's safe type system, parameterized queries, framework auto-escaping) from "makes exploitation harder" (ASLR, a WAF).
-
-**1.4**: Check whether similar code patterns exist elsewhere and are handled safely. Review test coverage, code review history (`git log -p`), and design documentation for this area.
-
-### Phase 2: Exploitability Verification
-
-**2.1**: Prove the attacker controls the data reaching the vulnerability. **Key pitfall**: assuming external data reaches the operation without tracing the actual path. Internal storage set by trusted components is not attacker-controlled.
-
-**2.2**: Create explicit algebraic proofs for bounds-related issues. Use the template in [evidence-templates.md](evidence-templates.md). Verify: IF validation_check_passes THEN bounds_guarantee_holds. Mark N/A when the bug is not about bounds or arithmetic.
-
-**2.3**: For race conditions, prove concurrent access is actually possible (two requests, two workers, two processes). **Key pitfall**: assuming races in single-threaded initialization or in code already serialized by a lock, transaction or unique constraint. Mark N/A when no concurrency is involved.
-
-**2.4**: Assess the full attack surface: input control, validation bypass paths, timing dependencies, and state manipulation.
-
-### Phase 3: Impact Assessment
-
-**3.1**: Distinguish real security impact (code execution, privilege escalation, information disclosure, unauthorized data changes, money loss) from operational robustness issues.
-
-**3.2**: Distinguish primary security controls from defense in depth. Failure of a defense-in-depth measure is not a vulnerability if primary protections remain intact.
-
-### Phase 4: PoC Creation
-
-**Always create a pseudocode PoC.** Create executable and/or unit test PoCs when feasible and allowed:
-
-1. **Pseudocode with data flow diagram** showing the attack path (always).
-2. **Executable PoC** in the target language (if feasible). In review and test steps it runs only from a temp directory outside the repository, never against live services or production data, and is deleted afterwards.
-3. **Unit test PoC** exercising the vulnerable path with crafted inputs (if feasible). Same rule: never add it to the repository under review. In a code or bugfix step it may become the regression test.
-
-See [evidence-templates.md](evidence-templates.md) for templates.
-
-**Negative PoC (4.4)**: show the gap between normal operation and the exploit path: which preconditions must hold for the vulnerability to trigger, and why they do not hold under normal conditions.
-
-### Phase 5: Devil's Advocate Review
-
-Before the verdict, challenge the claim systematically. Assume you are biased toward finding bugs and rating them as critical, and work against that bias.
-
-**Challenges arguing AGAINST the vulnerability:**
-
-1. What non-vulnerability explanations exist for this code pattern?
-2. How would the original developers justify this implementation?
-3. What crucial system architecture context might be missing?
-4. Am I seeing a vulnerability because the pattern "looks dangerous" rather than because it actually is?
-5. Even if validation looks insufficient, does it actually prevent the claimed condition?
-6. Am I incorrectly assuming attacker control over trusted data?
-7. Have I rigorously proven the mathematical condition for vulnerability can occur?
-8. Beyond theoretical possibility, is this practically exploitable?
-9. Am I confusing defense-in-depth failure with a primary security vulnerability?
-10. What compiler, runtime, OS or framework protections might prevent exploitation?
-11. Am I hallucinating this vulnerability? LLMs are biased toward seeing bugs everywhere and rating every finding as critical. Is this a real, exploitable issue, or am I pattern-matching on scary-looking code?
-
-**Challenges arguing FOR the vulnerability (false-negative protection):**
-
-12. Am I dismissing a real vulnerability because the exploit seems complex or unlikely?
-13. Am I inventing mitigations or validation logic that I have not verified in the actual source code? Re-read the code after reaching a conclusion.
-
-See [evidence-templates.md](evidence-templates.md) for the documentation template.
-
-## Gate Review
-
-Apply the six gates from [gate-reviews.md](gate-reviews.md) to reach a verdict.
-
-Part of the `false-positive-check` skill, adapted from https://github.com/trailofbits/skills/tree/main/plugins/fp-check (commit 5c99c03fb8a9ea568432ecbcba5d86e9f4ac2bf0), CC BY-SA 4.0. Changed: task tracking and plugin agents replaced by an in-order checklist (optionally the `code-explorer` agent), read-only rules for proofs of concept, N/A rules for 2.2 and 2.3, web examples of environment protections and impact.
+# Deep Verification
+
+Full verification for complex bugs. Use when routing in SKILL.md selects the deep path, or when standard verification escalates.
+
+## If Escalated from Standard
+
+1. Review all evidence gathered during the standard pass; do not repeat completed work.
+2. Identify which phases below are already satisfied by existing evidence.
+3. Continue from where standard verification left off.
+4. Keep and reference all prior findings.
+
+## Phases
+
+Work through the phases in order for each bug (Bug #N). Keep a checklist in your notes and mark a phase done only when you have concrete evidence for it. Sub-steps within a phase are independent; do them in any order, but never start a phase before the one it depends on is complete. If the bug path is large, you may hand a phase's data-flow tracing to the `code-explorer` agent, with the bug description and prior evidence in the prompt; you still judge the result.
+
+```
+Phase 1: Data flow analysis
+  1.1 Map trust boundaries and trace data flow
+  1.2 Research API contracts and safety guarantees
+  1.3 Environment protection analysis
+  1.4 Cross-reference analysis
+Phase 2: Exploitability verification (after phase 1)
+  2.1 Confirm attacker controls input data
+  2.2 Mathematical bounds verification (or N/A)
+  2.3 Race condition feasibility proof (or N/A)
+  2.4 Adversarial analysis (after 2.1 to 2.3)
+Phase 3: Impact assessment (after phase 2)
+  3.1 Demonstrate real security impact
+  3.2 Primary control vs defense in depth
+Phase 4: PoC creation (after phase 3)
+  4.1 Pseudocode PoC with data flow diagram
+  4.2 Executable PoC if feasible
+  4.3 Unit test PoC if feasible
+  4.4 Negative PoC: exploit preconditions
+  4.5 Verify the PoC demonstrates the vulnerability
+Phase 5: Devil's advocate review (after phase 4)
+Gate review (after phase 5)
+```
+
+Apply all 13 checklist items from [false-positive-patterns.md](false-positive-patterns.md) to each bug.
+
+## Phase Requirements
+
+The list above names every phase. Below are the key pitfalls and decision criteria for each: focus on what you might get wrong.
+
+### Phase 1: Data Flow Analysis
+
+**1.1**: Map trust boundaries (internal/trusted vs external/untrusted) and trace data from source to the alleged vulnerability. Apply class-specific verification from [bug-class-verification.md](bug-class-verification.md). **Key pitfall**: analyzing code in isolation without tracing the full validation chain. Conditional logic upstream may make the vulnerable code mathematically unreachable (see [false-positive-patterns.md](false-positive-patterns.md) items 1 and 1a).
+
+**1.2**: Check API contracts before claiming overflows or injections: many APIs have built-in protection (bounds checks, parameter binding, escaping) that prevents the alleged issue regardless of inputs.
+
+**1.3**: Before concluding vulnerability, verify that no compiler, runtime, OS, or framework protection prevents exploitation. Mitigations like ASLR and stack canaries raise the bar but do not eliminate the vulnerability itself. Distinguish "prevents exploitation entirely" (Rust's safe type system, parameterized queries, framework auto-escaping) from "makes exploitation harder" (ASLR, a WAF).
+
+**1.4**: Check whether similar code patterns exist elsewhere and are handled safely. Review test coverage, code review history (`git log -p`), and design documentation for this area.
+
+### Phase 2: Exploitability Verification
+
+**2.1**: Prove the attacker controls the data reaching the vulnerability. **Key pitfall**: assuming external data reaches the operation without tracing the actual path. Internal storage set by trusted components is not attacker-controlled.
+
+**2.2**: Create explicit algebraic proofs for bounds-related issues. Use the template in [evidence-templates.md](evidence-templates.md). Verify: IF validation_check_passes THEN bounds_guarantee_holds. Mark N/A when the bug is not about bounds or arithmetic.
+
+**2.3**: For race conditions, prove concurrent access is actually possible (two requests, two workers, two processes). **Key pitfall**: assuming races in single-threaded initialization or in code already serialized by a lock, transaction or unique constraint. Mark N/A when no concurrency is involved.
+
+**2.4**: Assess the full attack surface: input control, validation bypass paths, timing dependencies, and state manipulation.
+
+### Phase 3: Impact Assessment
+
+**3.1**: Distinguish real security impact (code execution, privilege escalation, information disclosure, unauthorized data changes, money loss) from operational robustness issues.
+
+**3.2**: Distinguish primary security controls from defense in depth. Failure of a defense-in-depth measure is not a vulnerability if primary protections remain intact.
+
+### Phase 4: PoC Creation
+
+**Always create a pseudocode PoC.** Create executable and/or unit test PoCs when feasible and allowed:
+
+1. **Pseudocode with data flow diagram** showing the attack path (always).
+2. **Executable PoC** in the target language (if feasible). In review and test steps it runs only from a temp directory outside the repository, never against live services or production data, and is deleted afterwards.
+3. **Unit test PoC** exercising the vulnerable path with crafted inputs (if feasible). Same rule: never add it to the repository under review. In a code or bugfix step it may become the regression test.
+
+See [evidence-templates.md](evidence-templates.md) for templates.
+
+**Negative PoC (4.4)**: show the gap between normal operation and the exploit path: which preconditions must hold for the vulnerability to trigger, and why they do not hold under normal conditions.
+
+### Phase 5: Devil's Advocate Review
+
+Before the verdict, challenge the claim systematically. Assume you are biased toward finding bugs and rating them as critical, and work against that bias.
+
+**Challenges arguing AGAINST the vulnerability:**
+
+1. What non-vulnerability explanations exist for this code pattern?
+2. How would the original developers justify this implementation?
+3. What crucial system architecture context might be missing?
+4. Am I seeing a vulnerability because the pattern "looks dangerous" rather than because it actually is?
+5. Even if validation looks insufficient, does it actually prevent the claimed condition?
+6. Am I incorrectly assuming attacker control over trusted data?
+7. Have I rigorously proven the mathematical condition for vulnerability can occur?
+8. Beyond theoretical possibility, is this practically exploitable?
+9. Am I confusing defense-in-depth failure with a primary security vulnerability?
+10. What compiler, runtime, OS or framework protections might prevent exploitation?
+11. Am I hallucinating this vulnerability? LLMs are biased toward seeing bugs everywhere and rating every finding as critical. Is this a real, exploitable issue, or am I pattern-matching on scary-looking code?
+
+**Challenges arguing FOR the vulnerability (false-negative protection):**
+
+12. Am I dismissing a real vulnerability because the exploit seems complex or unlikely?
+13. Am I inventing mitigations or validation logic that I have not verified in the actual source code? Re-read the code after reaching a conclusion.
+
+See [evidence-templates.md](evidence-templates.md) for the documentation template.
+
+## Gate Review
+
+Apply the six gates from [gate-reviews.md](gate-reviews.md) to reach a verdict.
+
+Part of the `false-positive-check` skill, adapted from https://github.com/trailofbits/skills/tree/main/plugins/fp-check (commit 5c99c03fb8a9ea568432ecbcba5d86e9f4ac2bf0), CC BY-SA 4.0. Changed: task tracking and plugin agents replaced by an in-order checklist (optionally the `code-explorer` agent), read-only rules for proofs of concept, N/A rules for 2.2 and 2.3, web examples of environment protections and impact.

package/library/skills/false-positive-check/references/evidence-templates.md

@@ -1,93 +1,93 @@
-# Evidence Templates
-
-Use these templates when documenting verification evidence for each bug.
-
-## Data Flow Documentation
-
-```
-Bug #N Data Flow Analysis
-Source: [exact location] - Trust Level: [trusted/untrusted]
-Path: Source → Validation1[file:line] → Transform[file:line] → Vulnerability[file:line]
-Validation Points:
-  - Check1: [condition] at [file:line] - [passes/fails/bypassed]
-  - Check2: [condition] at [file:line] - [passes/fails/bypassed]
-```
-
-## Mathematical Bounds Proof
-
-```
-Bug #N Mathematical Analysis
-Claim: Operation X is vulnerable to [overflow/underflow/bounds violation]
-Given Constraints: [list all validation conditions]
-
-Algebraic Proof:
-1. [first constraint from validation]
-2. [constant or known value]
-3. [derived inequality]
-...
-N. Therefore: [vulnerability confirmed/debunked] (Q.E.D.)
-
-Conclusion: [vulnerability is/is not mathematically possible]
-```
-
-**Example:**
-
-```
-Given: validation ensures (input_size >= MIN_SIZE)
-Given: MIN_SIZE = 16, header_size = 8
-Prove: (input_size - header_size) cannot underflow
-
-1. input_size >= MIN_SIZE             (from validation)
-2. MIN_SIZE = 16                      (constant)
-3. header_size = 8                    (constant)
-4. input_size >= 16                   (substitution of 1,2)
-5. input_size - 8 >= 16 - 8          (subtract header_size from both sides)
-6. input_size - header_size >= 8     (simplification)
-7. Therefore: underflow impossible    (Q.E.D.)
-```
-
-## Attacker Control Analysis
-
-```
-Bug #N Attacker Control Analysis
-Input Vector: [how attacker provides input]
-Control Level: [full/partial/none]
-Constraints: [what limits exist on attacker input]
-Reachability: [can attacker-controlled data reach vulnerable operation?]
-```
-
-## PoC - Pseudocode with Data Flow Diagram
-
-```
-PoC for Bug #N: [Brief Description]
-
-Data Flow Diagram:
-
-[External Input] → [Validation Point] → [Processing] → [Vulnerable Operation]
-     |                    |                   |                    |
-  Attacker           (May be bypassed)    (Transforms data)   (Unsafe operation)
-  Controlled              |                   |                    |
-     |                    v                   v                    v
-  [Malicious Data] → [Insufficient Check] → [Processed Data] → [Impact]
-
-PSEUDOCODE:
-function vulnerable_operation(user_data):
-    validation_result = weak_validation(user_data)  // Explain why this fails
-    processed_data = transform_data(user_data)      // Show transformation
-    unsafe_operation(processed_data)               // Show vulnerability trigger
-```
-
-## Devil's Advocate Review
-
-```
-Bug #N Devil's Advocate Review
-Vulnerability Claim: [brief description]
-
-For each devil's advocate question (7 in standard verification, 13 in deep verification), document your answer:
-Questions against the vulnerability: [answers]
-Questions for the vulnerability (false-negative protection): [answers]
-
-Final Assessment: [Vulnerability confirmed/debunked with reasoning]
-```
-
-Part of the `false-positive-check` skill, adapted from https://github.com/trailofbits/skills/tree/main/plugins/fp-check (commit 5c99c03fb8a9ea568432ecbcba5d86e9f4ac2bf0), CC BY-SA 4.0. Changed: punctuation and small wording changes for read-only runs.
+# Evidence Templates
+
+Use these templates when documenting verification evidence for each bug.
+
+## Data Flow Documentation
+
+```
+Bug #N Data Flow Analysis
+Source: [exact location] - Trust Level: [trusted/untrusted]
+Path: Source → Validation1[file:line] → Transform[file:line] → Vulnerability[file:line]
+Validation Points:
+  - Check1: [condition] at [file:line] - [passes/fails/bypassed]
+  - Check2: [condition] at [file:line] - [passes/fails/bypassed]
+```
+
+## Mathematical Bounds Proof
+
+```
+Bug #N Mathematical Analysis
+Claim: Operation X is vulnerable to [overflow/underflow/bounds violation]
+Given Constraints: [list all validation conditions]
+
+Algebraic Proof:
+1. [first constraint from validation]
+2. [constant or known value]
+3. [derived inequality]
+...
+N. Therefore: [vulnerability confirmed/debunked] (Q.E.D.)
+
+Conclusion: [vulnerability is/is not mathematically possible]
+```
+
+**Example:**
+
+```
+Given: validation ensures (input_size >= MIN_SIZE)
+Given: MIN_SIZE = 16, header_size = 8
+Prove: (input_size - header_size) cannot underflow
+
+1. input_size >= MIN_SIZE             (from validation)
+2. MIN_SIZE = 16                      (constant)
+3. header_size = 8                    (constant)
+4. input_size >= 16                   (substitution of 1,2)
+5. input_size - 8 >= 16 - 8          (subtract header_size from both sides)
+6. input_size - header_size >= 8     (simplification)
+7. Therefore: underflow impossible    (Q.E.D.)
+```
+
+## Attacker Control Analysis
+
+```
+Bug #N Attacker Control Analysis
+Input Vector: [how attacker provides input]
+Control Level: [full/partial/none]
+Constraints: [what limits exist on attacker input]
+Reachability: [can attacker-controlled data reach vulnerable operation?]
+```
+
+## PoC - Pseudocode with Data Flow Diagram
+
+```
+PoC for Bug #N: [Brief Description]
+
+Data Flow Diagram:
+
+[External Input] → [Validation Point] → [Processing] → [Vulnerable Operation]
+     |                    |                   |                    |
+  Attacker           (May be bypassed)    (Transforms data)   (Unsafe operation)
+  Controlled              |                   |                    |
+     |                    v                   v                    v
+  [Malicious Data] → [Insufficient Check] → [Processed Data] → [Impact]
+
+PSEUDOCODE:
+function vulnerable_operation(user_data):
+    validation_result = weak_validation(user_data)  // Explain why this fails
+    processed_data = transform_data(user_data)      // Show transformation
+    unsafe_operation(processed_data)               // Show vulnerability trigger
+```
+
+## Devil's Advocate Review
+
+```
+Bug #N Devil's Advocate Review
+Vulnerability Claim: [brief description]
+
+For each devil's advocate question (7 in standard verification, 13 in deep verification), document your answer:
+Questions against the vulnerability: [answers]
+Questions for the vulnerability (false-negative protection): [answers]
+
+Final Assessment: [Vulnerability confirmed/debunked with reasoning]
+```
+
+Part of the `false-positive-check` skill, adapted from https://github.com/trailofbits/skills/tree/main/plugins/fp-check (commit 5c99c03fb8a9ea568432ecbcba5d86e9f4ac2bf0), CC BY-SA 4.0. Changed: punctuation and small wording changes for read-only runs.