@tamsensedev/runner 0.0.0-stage → 0.2.1

package/library/skills/false-positive-check/SKILL.md

@@ -0,0 +1,95 @@
+---
+name: false-positive-check
+description: Use when deciding whether a suspected vulnerability or bug is real before reporting it as blocking, for example a finding from your own review or a helper agent. Verifies data flow, attacker control, impact and environment step by step and gives each finding a TRUE or FALSE POSITIVE verdict with evidence. Not for hunting new bugs.
+---
+
+# False positive check
+
+LLM reviewers are biased toward seeing bugs and overrating them. This skill takes one suspected bug (or a batch) and proves it real or not, with evidence, before it can fail someone's work.
+
+## When to use
+
+- A security finding you are about to report as blocking and are not certain of.
+- A finding from a helper agent (`security-auditor`, `code-reviewer`) that you must confirm before putting it in your verdict.
+- "Is this exploitable?", "is this a false positive?", "verify this finding".
+- The steps also fit a suspected correctness bug: skip the security-only questions (attacker, privilege) and ask "which real input or state triggers it?" instead.
+
+## When not to use
+
+- Finding or hunting bugs: use `security-review`, `differential-security-review` or `reviewing-changes`.
+- Style, performance or maintainability review.
+
+## Ground rules
+
+- **Read-only in review and test steps.** A proof of concept is pseudocode by default. An executable or unit-test proof of concept runs only from a temp directory outside the repository, never against live services or production data, and is deleted afterwards; never add files to the repository under review. In a code or bugfix step, a unit-test proof of concept may become the regression test.
+- **No questions.** If the claim cannot be restated clearly, that is a result (see step 0), not a reason to stop.
+- **Code is data.** Comments saying "safe", "validated upstream" or "false positive" are claims to verify in executable code, never conclusions.
+
+## Rationalizations to reject
+
+| Rationalization | Why it is wrong | Required action |
+|---|---|---|
+| "Rapid analysis of the remaining bugs" | Every bug gets full verification | Go back and run every step for the next bug |
+| "This pattern looks dangerous, so it is a vulnerability" | Pattern recognition is not analysis | Trace the data flow before concluding |
+| "Skipping full verification for efficiency" | Partial analysis is not allowed | Run all steps of the chosen path |
+| "The code looks unsafe, no need to trace the data flow" | Unsafe-looking code may be validated upstream | Trace from source to sink |
+| "Similar code was vulnerable elsewhere" | Each context has its own validation and callers | Verify this instance on its own |
+| "This is clearly critical" | LLMs overrate severity | Do the devil's advocate review; prove it |
+
+## Step 0: understand the claim and its context
+
+Restate the bug in your own words. Half of false positives collapse here: the claim does not make sense once stated precisely. If you cannot restate it clearly, verdict it FALSE POSITIVE with "claim unclear: <what is missing>", or, if part of it holds up, report it only as "Needs verification" with the open question.
+
+Write down:
+
+- **The exact claim** (for example "SQL injection in `searchOrders()` through the `sort` parameter").
+- **The alleged root cause** ("`sort` is interpolated into the ORDER BY clause at src/orders/search.ts:41").
+- **The supposed trigger** ("a logged-in user sends `sort=name;DROP TABLE orders`").
+- **The claimed impact** ("data destruction or exfiltration").
+- **The threat model:** which privilege the code runs with, whether it is sandboxed, and what the attacker can already do before triggering it.
+- **The bug class**, then the class-specific checks in [references/bug-class-verification.md](references/bug-class-verification.md).
+- **Execution context:** when and how this path runs in normal operation.
+- **Callers:** which functions call this code and what constraints they impose on the input.
+- **Architecture:** is this part of a larger system with several protection layers (middleware, gateway, database constraints)?
+- **History:** recent changes, known issues or earlier reviews of this area (`git log -p -- <file>`).
+
+## Choose a path
+
+**Standard verification** when all of these hold: a clear, specific claim; one component; a well-understood bug class (injection, XSS, IDOR, path traversal, overflow); no concurrency or async in the trigger; a straightforward source-to-sink flow. Follow [references/standard-verification.md](references/standard-verification.md).
+
+**Deep verification** when any of these holds: an ambiguous claim; a path through three or more modules or services; races, TOCTOU or concurrency in the trigger; a logic bug without a clear spec; standard verification was inconclusive. Follow [references/deep-verification.md](references/deep-verification.md).
+
+Default to standard: it has two escalation checkpoints that move you to deep when needed.
+
+Templates for the evidence: [references/evidence-templates.md](references/evidence-templates.md). Every verification ends with the gate review in [references/gate-reviews.md](references/gate-reviews.md) and the checklist in [references/false-positive-patterns.md](references/false-positive-patterns.md).
+
+## Batch triage
+
+When verifying several bugs:
+
+1. Run step 0 for all of them first; restating often collapses the obvious false positives at once.
+2. Choose the path for each bug separately.
+3. Do all standard-path bugs first, then the deep ones.
+4. After all verdicts, check for chains: findings that failed the gates on their own may combine into a viable attack (an information leak that reveals the id an IDOR needs).
+
+## Result
+
+For each bug, one line plus the evidence:
+
+```
+BUG #3 FALSE POSITIVE - SQL injection in src/orders/search.ts:41
+  Gate 2 (Reachability) FAIL: `sort` is checked against the allowlist ['name', 'date', 'total'] at src/orders/search.ts:30 before use.
+BUG #4 TRUE POSITIVE - invoices of other accounts readable through GET /invoices/:id (src/invoices/routes.ts:24)
+```
+
+Then the counts (X true positives, Y false positives) and the lists.
+
+How this feeds the verdict of a review or test step:
+
+- **TRUE POSITIVE:** report it, with severity per `security-review` (or `reviewing-changes` for correctness bugs). It may block.
+- **FALSE POSITIVE:** drop it. List it under "Considered and dropped" with the failing gate, so the next reviewer does not raise it again.
+- **Inconclusive** (a gate could not be decided from the code available): report as "Needs verification" with the exact open question. It does not block.
+
+If a `submit_verdict` tool is available and this check is the whole job, that is how the result is returned; otherwise give the result at the end of your reply.
+
+Adapted from https://github.com/trailofbits/skills/tree/main/plugins/fp-check (commit 5c99c03fb8a9ea568432ecbcba5d86e9f4ac2bf0), by Trail of Bits (Maciej Domanski), CC BY-SA 4.0. Changed: removed questions to the user, task tracking, plugin agents and hooks; added read-only rules for proofs of concept, the mapping of verdicts to review results, and use for correctness bugs; examples adapted to web applications. This skill folder, including its references, is licensed under CC BY-SA 4.0; see LICENSE.

package/library/skills/false-positive-check/references/bug-class-verification.md

@@ -0,0 +1,128 @@
+# Bug-Class-Specific Verification
+
+Different bug classes require different verification approaches. After classifying the bug in Step 0, apply the class-specific requirements below **in addition to** the generic verification phases.
+
+Jump to the relevant class:
+
+- [Memory Corruption](#memory-corruption): overflows, UAF, double-free, type confusion
+- [Logic Bugs](#logic-bugs): auth bypass, access control, state transitions, privilege escalation
+- [Race Conditions](#race-conditions): TOCTOU, data races, signal handling
+- [Integer Issues](#integer-issues): overflow, underflow, truncation, signedness
+- [Crypto Weaknesses](#crypto-weaknesses): weak algorithms, nonce reuse, timing channels
+- [Injection](#injection): SQL, XSS, command, template, path traversal
+- [Information Disclosure](#information-disclosure): uninitialized memory, leaks, side channels
+- [Denial of Service](#denial-of-service): algorithmic complexity, resource exhaustion, crashes
+- [Deserialization](#deserialization): object injection, gadget chains
+
+## Memory Corruption
+
+Buffer overflow, heap overflow, stack overflow, out-of-bounds read/write, use-after-free, double-free, type confusion.
+
+**Language safety check first:** Memory corruption in safe Rust, Go (without `unsafe.Pointer`/cgo), or managed languages (Java, C#, Python) is almost always a false positive: the type system or runtime prevents it. Verify whether the code is in an `unsafe` block (Rust), uses cgo/`unsafe.Pointer` (Go), or calls native code via JNI/P/Invoke. If the code is entirely in the safe subset, reject the memory corruption claim unless it involves a compiler bug or soundness hole.
+
+**Verify:**
+
+- What exactly gets corrupted? (which object, field, or memory region)
+- What is the corruption size and offset? Can the attacker control them?
+- Is the corruption a useful exploitation primitive (arbitrary read/write, vtable overwrite, function pointer overwrite) or just a crash?
+- What allocator is in use (glibc, tcmalloc, jemalloc, Windows heap)? Does it have hardening that blocks exploitation?
+- For UAF: trace the object lifetime: what frees it, what reuses the memory, can the attacker control the replacement object?
+- For type confusion: prove the type mismatch exists and that misinterpretation of the data leads to a useful primitive.
+
+## Logic Bugs
+
+Authentication bypass, access control errors, incorrect state transitions, confused deputy, privilege escalation through API misuse.
+
+**Verify:**
+
+- Check against the specification, RFC, or design docs: not just the code. Does the implementation match the intended behavior?
+- Map all state transitions. Can the system reach a state the developer didn't anticipate?
+- Identify implicit assumptions that are never enforced in code.
+- For auth bugs: verify ALL authentication/authorization paths, not just the one that appears broken. Is there a secondary check that catches it?
+- Logic bugs pass every bounds check and mathematical proof: don't let clean static analysis convince you it's a false positive.
+
+## Race Conditions
+
+TOCTOU, data races, signal handling races, concurrent state modification.
+
+**Verify:**
+
+- What is the actual race window? Is it nanoseconds or seconds?
+- Can the attacker widen the window (e.g., by stalling a thread with a slow NFS mount, large allocation, or CPU contention)?
+- Verify the threading model: what threads/processes can actually access this data concurrently?
+- Check all synchronization primitives in use: mutexes, atomics, RCU, lock-free structures.
+- For TOCTOU on filesystem: can the attacker control the path between check and use (symlink races)?
+
+## Integer Issues
+
+Overflow, underflow, truncation, signedness errors, wraparound.
+
+**Verify:**
+
+- What are the exact integer types and their ranges at every point in the computation?
+- Is the overflow signed (undefined behavior in C/C++: compiler may exploit this) or unsigned (defined wraparound)?
+- Trace the integer through all casts, conversions, and promotions. Where does truncation or sign extension occur?
+- After the integer issue occurs, is the resulting value actually used in a dangerous way (allocation size, array index, loop bound)?
+- Check if compiler warnings (`-Wconversion`, `-Wsign-compare`) flag this.
+
+## Crypto Weaknesses
+
+Weak algorithms, bad parameters, nonce reuse, padding oracle, insufficient randomness, timing side channels.
+
+**Verify:**
+
+- Check parameter choices against current standards (NIST, IETF) and known attacks. "AES-128" is fine; "DES" is not.
+- Verify randomness sources. Is the PRNG cryptographically secure? Is it properly seeded?
+- For nonce reuse: prove the same nonce can actually be used twice in practice, not just theoretically.
+- For timing side channels: is the code actually reachable by an attacker who can measure timing? Network jitter may make remote timing attacks impractical.
+- Compare the implementation against a reference implementation or test vectors from the spec.
+
+## Injection
+
+SQL injection, XSS, command injection, server-side template injection, path traversal, LDAP injection.
+
+**Verify:**
+
+- Trace attacker input from entry point to the sink (query, command, template, filesystem path). Is there any sanitization or escaping along the way?
+- Check if the framework provides automatic escaping (e.g., parameterized queries, template auto-escaping). If so, is it actually enabled and not bypassed?
+- For XSS: what context does the input land in (HTML body, attribute, JavaScript, URL)? Each requires different escaping.
+- For path traversal: is the path canonicalized before the access check? Can `../` or null bytes bypass validation?
+- Follow the actual payload through all intermediate processing (by reading the code, or by running it outside the repository): encoding, decoding, and transformation steps may neutralize or enable the payload.
+
+## Information Disclosure
+
+Uninitialized memory reads, error message leaks, timing side channels, padding oracles.
+
+**Verify:**
+
+- What specific data leaks? Not all leaks are equal: a stack leak revealing ASLR base or canary is critical; one revealing a static string is worthless.
+- Is the leaked data actually useful to an attacker for further exploitation (ASLR bypass, session tokens, crypto keys)?
+- For uninitialized memory: prove the memory is actually uninitialized at the point of read, not just potentially uninitialized on some code path.
+- For timing side channels: can the attacker make enough measurements with sufficient precision? What's the noise level?
+- For error messages: does the error path actually reach the attacker, or is it logged server-side only?
+
+## Denial of Service
+
+Algorithmic complexity, resource exhaustion, crash bugs, infinite loops, memory bombs.
+
+**Verify:**
+
+- What is the resource consumption ratio? Attacker sends X bytes, server consumes Y resources. Is the amplification meaningful?
+- Can the resource be reclaimed (connection closes, memory freed) or is it permanent exhaustion?
+- For algorithmic complexity: what is the actual worst-case input? Prove it triggers worst-case behavior, don't just claim O(n²).
+- For crash bugs: is the crash reliably triggerable, or does it depend on specific heap/stack layout?
+- Does the service restart automatically? A crash that causes a 100ms restart is different from one that requires manual intervention.
+
+## Deserialization
+
+Unsafe deserialization, object injection, gadget chain exploitation.
+
+**Verify:**
+
+- Does the attacker actually control the serialized data that reaches the deserialization call?
+- Does a usable gadget chain exist in the classpath/import graph? Without a gadget chain, unsafe deserialization is a design smell, not an exploitable bug.
+- What deserialization library and version is in use? Are there known gadget chains for it?
+- Are there type restrictions, allowlists, or look-ahead deserialization filters that block dangerous classes?
+- For language-specific: Java `ObjectInputStream`, Python `pickle`, PHP `unserialize`, .NET `BinaryFormatter` each have different exploitation characteristics.
+
+Part of the `false-positive-check` skill, adapted from https://github.com/trailofbits/skills/tree/main/plugins/fp-check (commit 5c99c03fb8a9ea568432ecbcba5d86e9f4ac2bf0), CC BY-SA 4.0. Changed: punctuation and small wording changes for read-only runs.

package/library/skills/false-positive-check/references/deep-verification.md

@@ -0,0 +1,113 @@
+# Deep Verification
+
+Full verification for complex bugs. Use when routing in SKILL.md selects the deep path, or when standard verification escalates.
+
+## If Escalated from Standard
+
+1. Review all evidence gathered during the standard pass; do not repeat completed work.
+2. Identify which phases below are already satisfied by existing evidence.
+3. Continue from where standard verification left off.
+4. Keep and reference all prior findings.
+
+## Phases
+
+Work through the phases in order for each bug (Bug #N). Keep a checklist in your notes and mark a phase done only when you have concrete evidence for it. Sub-steps within a phase are independent; do them in any order, but never start a phase before the one it depends on is complete. If the bug path is large, you may hand a phase's data-flow tracing to the `code-explorer` agent, with the bug description and prior evidence in the prompt; you still judge the result.
+
+```
+Phase 1: Data flow analysis
+  1.1 Map trust boundaries and trace data flow
+  1.2 Research API contracts and safety guarantees
+  1.3 Environment protection analysis
+  1.4 Cross-reference analysis
+Phase 2: Exploitability verification (after phase 1)
+  2.1 Confirm attacker controls input data
+  2.2 Mathematical bounds verification (or N/A)
+  2.3 Race condition feasibility proof (or N/A)
+  2.4 Adversarial analysis (after 2.1 to 2.3)
+Phase 3: Impact assessment (after phase 2)
+  3.1 Demonstrate real security impact
+  3.2 Primary control vs defense in depth
+Phase 4: PoC creation (after phase 3)
+  4.1 Pseudocode PoC with data flow diagram
+  4.2 Executable PoC if feasible
+  4.3 Unit test PoC if feasible
+  4.4 Negative PoC: exploit preconditions
+  4.5 Verify the PoC demonstrates the vulnerability
+Phase 5: Devil's advocate review (after phase 4)
+Gate review (after phase 5)
+```
+
+Apply all 13 checklist items from [false-positive-patterns.md](false-positive-patterns.md) to each bug.
+
+## Phase Requirements
+
+The list above names every phase. Below are the key pitfalls and decision criteria for each: focus on what you might get wrong.
+
+### Phase 1: Data Flow Analysis
+
+**1.1**: Map trust boundaries (internal/trusted vs external/untrusted) and trace data from source to the alleged vulnerability. Apply class-specific verification from [bug-class-verification.md](bug-class-verification.md). **Key pitfall**: analyzing code in isolation without tracing the full validation chain. Conditional logic upstream may make the vulnerable code mathematically unreachable (see [false-positive-patterns.md](false-positive-patterns.md) items 1 and 1a).
+
+**1.2**: Check API contracts before claiming overflows or injections: many APIs have built-in protection (bounds checks, parameter binding, escaping) that prevents the alleged issue regardless of inputs.
+
+**1.3**: Before concluding vulnerability, verify that no compiler, runtime, OS, or framework protection prevents exploitation. Mitigations like ASLR and stack canaries raise the bar but do not eliminate the vulnerability itself. Distinguish "prevents exploitation entirely" (Rust's safe type system, parameterized queries, framework auto-escaping) from "makes exploitation harder" (ASLR, a WAF).
+
+**1.4**: Check whether similar code patterns exist elsewhere and are handled safely. Review test coverage, code review history (`git log -p`), and design documentation for this area.
+
+### Phase 2: Exploitability Verification
+
+**2.1**: Prove the attacker controls the data reaching the vulnerability. **Key pitfall**: assuming external data reaches the operation without tracing the actual path. Internal storage set by trusted components is not attacker-controlled.
+
+**2.2**: Create explicit algebraic proofs for bounds-related issues. Use the template in [evidence-templates.md](evidence-templates.md). Verify: IF validation_check_passes THEN bounds_guarantee_holds. Mark N/A when the bug is not about bounds or arithmetic.
+
+**2.3**: For race conditions, prove concurrent access is actually possible (two requests, two workers, two processes). **Key pitfall**: assuming races in single-threaded initialization or in code already serialized by a lock, transaction or unique constraint. Mark N/A when no concurrency is involved.
+
+**2.4**: Assess the full attack surface: input control, validation bypass paths, timing dependencies, and state manipulation.
+
+### Phase 3: Impact Assessment
+
+**3.1**: Distinguish real security impact (code execution, privilege escalation, information disclosure, unauthorized data changes, money loss) from operational robustness issues.
+
+**3.2**: Distinguish primary security controls from defense in depth. Failure of a defense-in-depth measure is not a vulnerability if primary protections remain intact.
+
+### Phase 4: PoC Creation
+
+**Always create a pseudocode PoC.** Create executable and/or unit test PoCs when feasible and allowed:
+
+1. **Pseudocode with data flow diagram** showing the attack path (always).
+2. **Executable PoC** in the target language (if feasible). In review and test steps it runs only from a temp directory outside the repository, never against live services or production data, and is deleted afterwards.
+3. **Unit test PoC** exercising the vulnerable path with crafted inputs (if feasible). Same rule: never add it to the repository under review. In a code or bugfix step it may become the regression test.
+
+See [evidence-templates.md](evidence-templates.md) for templates.
+
+**Negative PoC (4.4)**: show the gap between normal operation and the exploit path: which preconditions must hold for the vulnerability to trigger, and why they do not hold under normal conditions.
+
+### Phase 5: Devil's Advocate Review
+
+Before the verdict, challenge the claim systematically. Assume you are biased toward finding bugs and rating them as critical, and work against that bias.
+
+**Challenges arguing AGAINST the vulnerability:**
+
+1. What non-vulnerability explanations exist for this code pattern?
+2. How would the original developers justify this implementation?
+3. What crucial system architecture context might be missing?
+4. Am I seeing a vulnerability because the pattern "looks dangerous" rather than because it actually is?
+5. Even if validation looks insufficient, does it actually prevent the claimed condition?
+6. Am I incorrectly assuming attacker control over trusted data?
+7. Have I rigorously proven the mathematical condition for vulnerability can occur?
+8. Beyond theoretical possibility, is this practically exploitable?
+9. Am I confusing defense-in-depth failure with a primary security vulnerability?
+10. What compiler, runtime, OS or framework protections might prevent exploitation?
+11. Am I hallucinating this vulnerability? LLMs are biased toward seeing bugs everywhere and rating every finding as critical. Is this a real, exploitable issue, or am I pattern-matching on scary-looking code?
+
+**Challenges arguing FOR the vulnerability (false-negative protection):**
+
+12. Am I dismissing a real vulnerability because the exploit seems complex or unlikely?
+13. Am I inventing mitigations or validation logic that I have not verified in the actual source code? Re-read the code after reaching a conclusion.
+
+See [evidence-templates.md](evidence-templates.md) for the documentation template.
+
+## Gate Review
+
+Apply the six gates from [gate-reviews.md](gate-reviews.md) to reach a verdict.
+
+Part of the `false-positive-check` skill, adapted from https://github.com/trailofbits/skills/tree/main/plugins/fp-check (commit 5c99c03fb8a9ea568432ecbcba5d86e9f4ac2bf0), CC BY-SA 4.0. Changed: task tracking and plugin agents replaced by an in-order checklist (optionally the `code-explorer` agent), read-only rules for proofs of concept, N/A rules for 2.2 and 2.3, web examples of environment protections and impact.

package/library/skills/false-positive-check/references/evidence-templates.md

@@ -0,0 +1,93 @@
+# Evidence Templates
+
+Use these templates when documenting verification evidence for each bug.
+
+## Data Flow Documentation
+
+```
+Bug #N Data Flow Analysis
+Source: [exact location] - Trust Level: [trusted/untrusted]
+Path: Source → Validation1[file:line] → Transform[file:line] → Vulnerability[file:line]
+Validation Points:
+  - Check1: [condition] at [file:line] - [passes/fails/bypassed]
+  - Check2: [condition] at [file:line] - [passes/fails/bypassed]
+```
+
+## Mathematical Bounds Proof
+
+```
+Bug #N Mathematical Analysis
+Claim: Operation X is vulnerable to [overflow/underflow/bounds violation]
+Given Constraints: [list all validation conditions]
+
+Algebraic Proof:
+1. [first constraint from validation]
+2. [constant or known value]
+3. [derived inequality]
+...
+N. Therefore: [vulnerability confirmed/debunked] (Q.E.D.)
+
+Conclusion: [vulnerability is/is not mathematically possible]
+```
+
+**Example:**
+
+```
+Given: validation ensures (input_size >= MIN_SIZE)
+Given: MIN_SIZE = 16, header_size = 8
+Prove: (input_size - header_size) cannot underflow
+
+1. input_size >= MIN_SIZE             (from validation)
+2. MIN_SIZE = 16                      (constant)
+3. header_size = 8                    (constant)
+4. input_size >= 16                   (substitution of 1,2)
+5. input_size - 8 >= 16 - 8          (subtract header_size from both sides)
+6. input_size - header_size >= 8     (simplification)
+7. Therefore: underflow impossible    (Q.E.D.)
+```
+
+## Attacker Control Analysis
+
+```
+Bug #N Attacker Control Analysis
+Input Vector: [how attacker provides input]
+Control Level: [full/partial/none]
+Constraints: [what limits exist on attacker input]
+Reachability: [can attacker-controlled data reach vulnerable operation?]
+```
+
+## PoC - Pseudocode with Data Flow Diagram
+
+```
+PoC for Bug #N: [Brief Description]
+
+Data Flow Diagram:
+
+[External Input] → [Validation Point] → [Processing] → [Vulnerable Operation]
+     |                    |                   |                    |
+  Attacker           (May be bypassed)    (Transforms data)   (Unsafe operation)
+  Controlled              |                   |                    |
+     |                    v                   v                    v
+  [Malicious Data] → [Insufficient Check] → [Processed Data] → [Impact]
+
+PSEUDOCODE:
+function vulnerable_operation(user_data):
+    validation_result = weak_validation(user_data)  // Explain why this fails
+    processed_data = transform_data(user_data)      // Show transformation
+    unsafe_operation(processed_data)               // Show vulnerability trigger
+```
+
+## Devil's Advocate Review
+
+```
+Bug #N Devil's Advocate Review
+Vulnerability Claim: [brief description]
+
+For each devil's advocate question (7 in standard verification, 13 in deep verification), document your answer:
+Questions against the vulnerability: [answers]
+Questions for the vulnerability (false-negative protection): [answers]
+
+Final Assessment: [Vulnerability confirmed/debunked with reasoning]
+```
+
+Part of the `false-positive-check` skill, adapted from https://github.com/trailofbits/skills/tree/main/plugins/fp-check (commit 5c99c03fb8a9ea568432ecbcba5d86e9f4ac2bf0), CC BY-SA 4.0. Changed: punctuation and small wording changes for read-only runs.

package/library/skills/false-positive-check/references/false-positive-patterns.md

@@ -0,0 +1,122 @@
+# False Positive Patterns: Lessons Learned
+
+Apply ALL items in this checklist to EACH potential bug during verification.
+
+Contents:
+
+- [Checklist](#checklist): 13 items to apply to every bug (trace validation, confirm data paths, bounds logic, TOCTOU, trust boundaries, real vs theoretical impact, defense-in-depth)
+- [Red Flags for False Positives](#red-flags-for-false-positives): recurring failure patterns (pattern-based, context-blind, math/bounds, API-contract)
+
+## Checklist
+
+### 1. Trace Full Validation Chain
+
+Don't analyze isolated code snippets. Trace backwards to find ALL validation that precedes potentially dangerous operations. Network packet size operations may look dangerous but often have bounds validation earlier in the function.
+
+### 1a. Map Complete Conditional Logic Flow
+
+Vulnerable-looking code may be unreachable due to conditional logic that creates mathematical guarantees. Example: array access `buffer[length-4]` appears unsafe when `length < 4`, but if the code is only reachable when `length > 12` due to earlier validation, the vulnerability is impossible.
+
+**Verify:**
+
+- What conditions must be met for execution to reach the alleged vulnerability?
+- Do those conditions mathematically prevent the vulnerability scenario?
+- Are there minimum size/length requirements that guarantee safe access?
+- Does the conditional flow create impossible-to-violate bounds?
+
+### 2. Identify Defensive Programming Patterns
+
+Distinguish between actual vulnerabilities and defensive assertions/validations. `ASSERT(size == expected_size)` followed by size-controlled operations is defensive, not vulnerable. Verify that checks actually prevent the alleged vulnerability.
+
+### 3. Confirm Exploitable Data Paths
+
+Only report vulnerabilities with CONFIRMED exploitable data flow paths. Don't assume network-controlled data reaches dangerous functions without tracing the actual path step by step.
+
+### 4. Understand Data Source Context
+
+Distinguish between data sources and their trust levels. API return values, compile-time constants, and network data have different risk profiles. Determine the actual source and whether it is attacker-controlled.
+
+### 5. Analyze Bounds Validation Logic
+
+Look for mathematical relationships between validation checks and subsequent operations. If `packet_size >= MIN_SIZE` is checked and `MIN_SIZE >= sizeof(header)`, then `packet_size - sizeof(header)` cannot underflow.
+
+### 6. Verify TOCTOU Claims
+
+Time-of-check-time-of-use issues require proof that the checked value can change between check and use. If a size is checked and immediately used in the same function with no external modification possible, there is no TOCTOU.
+
+### 7. Understand API Contract and Trust Boundaries
+
+Always understand API contracts before claiming buffer overflows. Some APIs have built-in bounds protection and cannot write beyond the buffer regardless of input parameters.
+
+### 8. Distinguish Internal Storage from External Input
+
+Internal storage systems (configuration stores, registries) are controlled by trusted components, not attackers. Values set during installation by trusted components are not attacker-controlled.
+
+### 9. Don't Confuse Pattern Recognition with Vulnerability Analysis
+
+Code patterns that "look vulnerable" may be safely implemented due to context and API contracts. Size parameters being modified doesn't mean buffer overflow if the API prevents writing beyond bounds.
+
+### 10. Verify Concurrent Access is Actually Possible
+
+Don't assume race conditions exist without proving concurrent access patterns. Single-threaded initialization contexts cannot have race conditions. Verify the threading model and synchronization mechanisms.
+
+### 11. Assess Real vs Theoretical Security Impact
+
+Focus on vulnerabilities with actual security impact. Storage failure for non-critical data is an operational issue, not a security vulnerability. Ask: would this lead to code execution, privilege escalation, or information disclosure?
+
+### 12. Understand Defense-in-Depth vs Primary Controls
+
+Failure of defense-in-depth mechanisms is not always a vulnerability if primary protections exist. Token cleanup failure is not critical if tokens are single-use by design at the server.
+
+### 13. Apply the Checklist Rigorously, Not Superficially
+
+Having a checklist doesn't prevent false positives if it isn't applied systematically. For EVERY potential vulnerability, work through ALL checklist items before concluding.
+
+---
+
+## Red Flags for False Positives
+
+### Pattern-Based False Positives
+
+- Reporting vulnerabilities in validation/bounds-checking code itself
+- Claiming TOCTOU without proving the value can change
+- Ignoring preceding validation logic
+- Assuming network data reaches operations without tracing the path
+- Confusing defensive programming (assertions/checks) with vulnerabilities
+- Analyzing vulnerable-looking patterns without tracing conditional logic that controls reachability
+- Reporting "vulnerabilities" in error handling or cleanup code
+- Flagging size calculations without understanding mathematical constraints
+- Identifying "dangerous" functions without checking if inputs are bounded
+- Claiming buffer overflows in fixed-size operations with compile-time bounds
+- Reporting race conditions in single-threaded or synchronized contexts
+
+### Context-Blind Analysis False Positives
+
+- Analyzing code snippets without understanding broader system design
+- Ignoring architectural guarantees (single-writer, trusted input sources)
+- Missing that "vulnerable" code is unreachable due to earlier validation
+- Confusing debug/development code paths with production paths
+- Reporting issues in code that only runs during trusted installation/setup
+- Flagging theoretical issues that cannot occur due to system architecture
+- Missing that alleged vulnerabilities are prevented by framework or language guarantees
+- Reporting issues in test-only or debug-only code paths as production vulnerabilities
+
+### Mathematical/Bounds Analysis False Positives
+
+- Reporting integer underflow without proving the mathematical condition can occur
+- Claiming buffer overflow when bounds are mathematically guaranteed by validation
+- Missing that conditional logic creates mathematical impossibility of vulnerable conditions
+- Reporting off-by-one errors without checking if loop bounds prevent the condition
+- Claiming memory corruption when allocation sizes are verified sufficient
+- Reporting arithmetic overflow without checking if input ranges prevent the condition
+
+### API Contract Misunderstanding False Positives
+
+- Claiming buffer overflows when APIs have built-in bounds checking
+- Reporting memory corruption for APIs that manage their own memory safely
+- Missing that return values are already validated by the API contract
+- Confusing API parameter modification with vulnerability when API prevents unsafe modification
+- Reporting issues explicitly handled by the API's safety guarantees
+- Missing that seemingly dangerous operations are safe due to API implementation details
+
+Part of the `false-positive-check` skill, adapted from https://github.com/trailofbits/skills/tree/main/plugins/fp-check (commit 5c99c03fb8a9ea568432ecbcba5d86e9f4ac2bf0), CC BY-SA 4.0. Changed: punctuation and small wording changes for read-only runs.

package/library/skills/false-positive-check/references/gate-reviews.md

@@ -0,0 +1,36 @@
+# Gate Reviews and Verdicts
+
+Before reporting ANY bug as a vulnerability, all six gate reviews must pass. Evaluate them at the end, after all phases are complete:
+
+| Gate | Criterion | Pass | Fail |
+|------|-----------|------|------|
+| **1. Process** | All phases completed with documented evidence | Evidence exists for every phase | Phases lack concrete evidence |
+| **2. Reachability** | Attacker can reach and control data at the vulnerability | Clear evidence of attacker-controlled path + PoC confirms | Cannot demonstrate attacker control or reachability |
+| **3. Real Impact** | Exploitation leads to code execution, privilege escalation, information disclosure, unauthorized changes to data, or loss of money | Direct impact with concrete scenarios | Only operational robustness issue |
+| **4. PoC Validation** | PoC (pseudocode, executable, or unit test) demonstrates the attack path | Shows attacker control, trigger, and impact | PoC fails to show attack path or impact |
+| **5. Math Bounds** | Mathematical analysis confirms vulnerable condition is possible (N/A, counted as pass, when the bug is not about bounds or arithmetic) | Algebraic proof shows condition is possible | Math proves validation prevents it |
+| **6. Environment** | No environmental protections entirely prevent exploitation | Protections do not eliminate vulnerability | Environmental protections block it entirely |
+
+For a suspected correctness bug rather than a vulnerability: gate 2 asks whether a realistic input or state reaches the faulty code, and gate 3 whether the outcome is a wrong result, lost or corrupted data, or a crash that users hit.
+
+## Verdict Format
+
+- **TRUE POSITIVE**: All gate reviews pass → `BUG #N TRUE POSITIVE - [brief vulnerability description]`
+- **FALSE POSITIVE**: Any gate review fails → `BUG #N FALSE POSITIVE - [brief reason for rejection]`
+- **INCONCLUSIVE**: A gate cannot be decided from the code available (the caller is in another repository, the configuration is not in the repo) → `BUG #N NEEDS VERIFICATION - [the exact open question]`. Reported as a note, never as blocking.
+
+If any phase fails verification, document the failure with evidence and continue all remaining phases. Issue the FALSE POSITIVE verdict only after all phases are complete.
+
+## Example Verdict
+
+```
+BUG #3 FALSE POSITIVE - Integer underflow in packet_handler.c:142
+  Gate 5 (Math Bounds) FAIL: validation at line 98 ensures packet_size >= 16,
+  making (packet_size - header_size) >= 8. Underflow is mathematically impossible.
+
+BUG #5 FALSE POSITIVE - IDOR in src/api/files.ts:19
+  Gate 2 (Reachability) FAIL: the route is mounted under accountRouter (src/app.ts:52),
+  whose middleware loads the file with the session's account id; other accounts' files return 404.
+```
+
+Part of the `false-positive-check` skill, adapted from https://github.com/trailofbits/skills/tree/main/plugins/fp-check (commit 5c99c03fb8a9ea568432ecbcba5d86e9f4ac2bf0), CC BY-SA 4.0. Changed: wider impact list for web applications, N/A rule for gate 5, correctness-bug reading of gates 2 and 3, inconclusive verdict, second example.