@cubis/foundry 0.3.10

package/Ai Agent Workflow/skills/fix-review/references/finding-matching.md

@@ -0,0 +1,298 @@
+# Finding Matching Strategies
+
+Techniques for matching security findings to code commits.
+
+## Overview
+
+Matching findings to commits requires multiple approaches since:
+- Commit messages may not reference finding IDs
+- Findings may span multiple files
+- Multiple commits may partially address a single finding
+- A single commit may address multiple findings
+
+---
+
+## Matching Approaches
+
+### 1. Direct ID Reference
+
+Search commit messages for finding IDs:
+
+```bash
+# Search for TOB-style IDs in commit messages
+git log <source>..<target> --grep="TOB-" --oneline
+
+# Search for generic finding references
+git log <source>..<target> --grep="[Ff]inding" --oneline
+git log <source>..<target> --grep="[Ff]ix" --oneline
+```
+
+**Confidence:** High when found, but many commits lack explicit references.
+
+### 2. File Path Matching
+
+Match findings by affected files:
+
+```bash
+# Get files changed in commit range
+git diff <source>..<target> --name-only
+
+# Compare with files mentioned in finding
+# Finding: "The vulnerability exists in contracts/Vault.sol"
+# Check: Does any commit modify contracts/Vault.sol?
+```
+
+**Workflow:**
+1. Extract file paths from finding description
+2. List changed files in commit range
+3. Identify commits touching those files
+4. Analyze those commits in detail
+
+### 3. Function/Symbol Matching
+
+Match by function or variable names:
+
+```bash
+# Search for function name in diffs
+git log <source>..<target> -p | grep -A5 -B5 "function withdraw"
+
+# Search for specific patterns
+git log <source>..<target> -S "functionName" --oneline
+```
+
+**Extract symbols from findings:**
+- Function names: `withdraw()`, `transfer()`, `validateInput()`
+- Variable names: `balance`, `owner`, `allowance`
+- Contract/class names: `Vault`, `TokenManager`
+
+### 4. Code Pattern Matching
+
+Match by vulnerability pattern:
+
+```bash
+# Finding mentions "missing require statement"
+# Search for added require statements
+git diff <source>..<target> | grep "^+" | grep "require"
+
+# Finding mentions "reentrancy"
+# Search for state changes and external calls
+git diff <source>..<target> | grep -E "(\.call|\.transfer|\.send)"
+```
+
+---
+
+## Matching Workflow
+
+### Step 1: Extract Finding Metadata
+
+For each finding, extract:
+
+| Field | Example |
+|-------|---------|
+| ID | TOB-CLIENT-1 |
+| Title | Missing access control in withdraw() |
+| Severity | High |
+| Files | contracts/Vault.sol:L45-L67 |
+| Functions | withdraw(), _validateCaller() |
+| Pattern | Access control |
+| Recommendation | Add onlyOwner modifier |
+
+### Step 2: Search for Direct Matches
+
+```bash
+# Check for ID in commit messages
+git log <source>..<target> --grep="TOB-CLIENT-1" --oneline
+
+# Check for title keywords
+git log <source>..<target> --grep="access control" --oneline
+git log <source>..<target> --grep="withdraw" --oneline
+```
+
+### Step 3: Identify Relevant Commits
+
+For each file mentioned in the finding:
+
+```bash
+# Get commits that modified the file
+git log <source>..<target> --oneline -- contracts/Vault.sol
+
+# Get the diff for that file
+git diff <source>..<target> -- contracts/Vault.sol
+```
+
+### Step 4: Analyze Fix Quality
+
+For each potentially matching commit:
+
+1. **Read the full diff** - Understand what changed
+2. **Compare with recommendation** - Does the fix follow the suggested approach?
+3. **Check completeness** - Are all instances of the vulnerability fixed?
+4. **Verify correctness** - Is the fix itself correct (no logic errors)?
+
+---
+
+## Status Assignment Criteria
+
+### FIXED
+
+Assign when:
+- Code change directly addresses the root cause
+- Fix follows the report's recommendation (or equivalent)
+- All instances of the vulnerability are addressed
+- No obvious issues with the fix itself
+
+**Evidence required:**
+- Commit hash
+- File and line numbers
+- Brief explanation of how fix addresses the finding
+
+### PARTIALLY_FIXED
+
+Assign when:
+- Some instances fixed, others remain
+- Fix addresses symptoms but not root cause
+- Fix is incomplete (missing edge cases)
+- Fix works but doesn't follow best practice
+
+**Evidence required:**
+- What was fixed (with commit hash)
+- What remains unfixed
+- Specific gaps in the fix
+
+### NOT_ADDRESSED
+
+Assign when:
+- No commits modify relevant files
+- Changes to relevant files don't address the finding
+- Finding relates to architecture/design not changed
+
+**Evidence required:**
+- Confirmation that relevant files were checked
+- Brief explanation of why no fix was found
+
+### CANNOT_DETERMINE
+
+Assign when:
+- Finding is ambiguous
+- Code changes are unclear
+- Requires runtime analysis to verify
+- Need additional context from developers
+
+**Evidence required:**
+- What was analyzed
+- Specific questions that need answers
+- Suggested next steps
+
+---
+
+## Complex Scenarios
+
+### Multiple Commits for One Finding
+
+When several commits contribute to fixing a single finding:
+
+1. List all relevant commits
+2. Analyze each contribution
+3. Determine if combined effect is FIXED or PARTIALLY_FIXED
+4. Document each commit's contribution
+
+**Example:**
+```
+TOB-XXX-1: Access control vulnerability in withdraw()
+
+Commits:
+- abc123: Added onlyOwner modifier
+- def456: Added balance check
+- ghi789: Added event emission
+
+Combined: FIXED
+- abc123 addresses the core access control issue
+- def456 adds defense in depth
+- ghi789 improves auditability
+```
+
+### One Commit for Multiple Findings
+
+When a single commit addresses multiple findings:
+
+1. Analyze the commit once
+2. Map specific changes to each finding
+3. Assign status to each finding individually
+4. Reference the same commit in multiple findings
+
+### Interacting Findings
+
+When findings are related and fixes may interact:
+
+1. Identify the relationship
+2. Analyze fixes together
+3. Check for conflicts or regressions
+4. Document the interaction
+
+**Example:**
+```
+TOB-XXX-1: Reentrancy in withdraw()
+TOB-XXX-2: Missing balance validation
+
+These interact: A reentrancy fix might break the balance check
+Analysis: Commit abc123 uses checks-effects-interactions pattern
+Result: Both findings addressed without conflict
+```
+
+---
+
+## Handling Ambiguity
+
+### When Finding Description is Vague
+
+1. Search for related patterns in the codebase
+2. Look for commit messages mentioning the issue
+3. Check if any changes seem security-related
+4. Mark as CANNOT_DETERMINE if unclear
+
+### When Multiple Interpretations Exist
+
+1. Document both interpretations
+2. Analyze against both
+3. Note which interpretation the fix addresses
+4. Flag for developer clarification if needed
+
+### When Fix Differs from Recommendation
+
+The fix may be valid even if different from the recommendation:
+
+1. Understand the recommended approach
+2. Analyze the actual fix
+3. Determine if it addresses the root cause
+4. Mark as FIXED if effective, note the difference
+
+---
+
+## Git Commands Reference
+
+```bash
+# List commits in range
+git log <source>..<target> --oneline
+
+# Search commit messages
+git log <source>..<target> --grep="pattern" --oneline
+
+# Get files changed
+git diff <source>..<target> --name-only
+
+# Get full diff
+git diff <source>..<target>
+
+# Get diff for specific file
+git diff <source>..<target> -- path/to/file
+
+# Search for code changes
+git log <source>..<target> -S "code_pattern" --oneline
+
+# Get commit details
+git show <commit> --stat
+git show <commit> -p
+
+# Blame specific lines
+git blame <commit> -- path/to/file
+```

package/Ai Agent Workflow/skills/fix-review/references/report-parsing.md

@@ -0,0 +1,398 @@
+# Report Parsing Strategies
+
+Parsing security audit reports in various formats.
+
+## Overview
+
+Security reports come in multiple formats. This guide covers parsing strategies for each format and handling special cases like Google Drive URLs.
+
+---
+
+## Trail of Bits Format
+
+Trail of Bits reports follow a consistent structure.
+
+### Structure
+
+```
+1. Executive Summary
+2. Project Dashboard
+3. Engagement Goals
+4. Coverage
+5. Automated Testing
+6. Findings Overview
+7. Detailed Findings
+   - Each finding starts on new page
+   - Header table with ID, title, severity, type, target
+   - Description, Exploit Scenario, Recommendations
+8. Appendices
+```
+
+### Finding Identification
+
+Each finding has a header table:
+
+| Field | Format |
+|-------|--------|
+| ID | `TOB-[CLIENT]-[NUMBER]` (e.g., TOB-ACME-1) |
+| Title | Descriptive title |
+| Severity | Informational, Low, Medium, High |
+| Difficulty | Low, Medium, High, Undetermined |
+| Type | Access Controls, Cryptography, Data Validation, etc. |
+| Target | File path(s) |
+
+### Extraction Pattern
+
+```
+1. Locate "Detailed Findings" section
+2. For each finding, extract:
+   - ID: Match pattern /TOB-[A-Z]+-[0-9]+/
+   - Title: Text following ID in header
+   - Severity: From header table
+   - Target: File paths from header table
+   - Description: Content after "Description" heading
+   - Recommendations: Content after "Recommendations" heading
+```
+
+### Example Finding
+
+```markdown
+## TOB-ACME-1: Missing access control in withdraw function
+
+| Field | Value |
+|-------|-------|
+| ID | TOB-ACME-1 |
+| Severity | High |
+| Difficulty | Low |
+| Type | Access Controls |
+| Target | contracts/Vault.sol |
+
+### Description
+
+The `withdraw` function in `Vault.sol` lacks access control...
+
+### Recommendations
+
+Short term, add the `onlyOwner` modifier...
+```
+
+---
+
+## Generic Report Formats
+
+### Numbered Findings
+
+Reports with numbered findings (Finding 1, Finding 2, etc.):
+
+```
+Pattern: /Finding\s+[0-9]+:?\s+(.+)/
+         /[0-9]+\.\s+(.+)/
+         /#[0-9]+\s+(.+)/
+```
+
+Extract:
+- Number as ID
+- Following text as title
+- Look for severity keywords nearby
+
+### Severity-Based Sections
+
+Reports organized by severity:
+
+```
+## Critical
+### Finding title
+...
+
+## High
+### Another finding
+...
+```
+
+Extract:
+- Section heading as severity
+- Sub-headings as finding titles
+- Generate IDs (CRITICAL-1, HIGH-1, etc.)
+
+### Table-Based Findings
+
+Reports with findings in tables:
+
+```markdown
+| ID | Title | Severity | Status |
+|----|-------|----------|--------|
+| V-01 | SQL Injection | High | Open |
+| V-02 | XSS in search | Medium | Open |
+```
+
+Extract by parsing table structure.
+
+### JSON Format
+
+Reports in JSON structure:
+
+```json
+{
+  "findings": [
+    {
+      "id": "VULN-001",
+      "title": "SQL Injection",
+      "severity": "high",
+      "description": "...",
+      "files": ["app/db.py"]
+    }
+  ]
+}
+```
+
+Parse directly from JSON structure.
+
+---
+
+## Format Detection
+
+When report format is unknown:
+
+### Step 1: Check for TOB Format
+
+```
+Search for: "TOB-" followed by letters and numbers
+If found: Use TOB parsing
+```
+
+### Step 2: Check for JSON
+
+```
+If file extension is .json or content starts with '{':
+  Parse as JSON
+  Look for "findings" array
+```
+
+### Step 3: Check for Markdown Structure
+
+```
+Search for: "## Finding" or "### Finding"
+Search for: Severity headings (Critical, High, Medium, Low)
+Search for: Numbered patterns (1., 2., or Finding 1, Finding 2)
+```
+
+### Step 4: Fall Back to Keyword Extraction
+
+```
+Search for severity keywords: critical, high, medium, low, informational
+Search for vulnerability keywords: vulnerability, issue, bug, flaw
+Extract surrounding context as findings
+```
+
+---
+
+## Google Drive Handling
+
+When a Google Drive URL is provided and WebFetch fails (permissions, redirect):
+
+### Step 1: Detect Google Drive URL
+
+```
+Pattern: https://drive.google.com/file/d/[FILE_ID]/...
+         https://docs.google.com/document/d/[DOC_ID]/...
+         https://drive.google.com/open?id=[FILE_ID]
+```
+
+### Step 2: Extract File ID
+
+```bash
+# From /file/d/ URLs
+FILE_ID=$(echo "$URL" | grep -oP 'file/d/\K[^/]+')
+
+# From /document/d/ URLs
+FILE_ID=$(echo "$URL" | grep -oP 'document/d/\K[^/]+')
+
+# From ?id= URLs
+FILE_ID=$(echo "$URL" | grep -oP 'id=\K[^&]+')
+```
+
+### Step 3: Check for gdrive CLI
+
+```bash
+# Check if gdrive is installed
+if command -v gdrive &> /dev/null; then
+    # Check if gdrive is configured (has auth)
+    if gdrive about &> /dev/null; then
+        echo "gdrive available and configured"
+    else
+        echo "gdrive installed but not configured"
+    fi
+else
+    echo "gdrive not installed"
+fi
+```
+
+### Step 4: Download with gdrive
+
+If gdrive is available and configured:
+
+```bash
+# Download to temp directory
+gdrive files download "$FILE_ID" --path /tmp/
+
+# Find the downloaded file
+DOWNLOADED=$(ls -t /tmp/ | head -1)
+
+# Read the file
+cat "/tmp/$DOWNLOADED"
+```
+
+### Step 5: User Instructions (if gdrive unavailable)
+
+If gdrive is not available or not configured:
+
+```
+Unable to access the Google Drive URL directly. Please:
+
+1. Open the URL in your browser
+2. Download the file:
+   - For Google Docs: File → Download → Markdown (.md)
+   - For PDFs: Click download button
+3. Provide the local file path
+
+Alternatively, install and configure gdrive:
+   brew install gdrive
+   gdrive about  # Follow auth prompts
+```
+
+---
+
+## File Format Handling
+
+### PDF Files
+
+Claude can read PDFs directly using the Read tool:
+
+```
+Read /path/to/report.pdf
+```
+
+For large PDFs, process section by section:
+1. Read table of contents/overview
+2. Locate "Findings" section
+3. Read findings section in detail
+
+### Markdown Files
+
+Read directly:
+
+```
+Read /path/to/report.md
+```
+
+### HTML Files
+
+Read and parse:
+
+```
+Read /path/to/report.html
+```
+
+Extract text content, ignoring HTML tags.
+
+### JSON Files
+
+Read and parse as structured data:
+
+```
+Read /path/to/report.json
+```
+
+Access fields directly from JSON structure.
+
+---
+
+## Extraction Output Format
+
+Regardless of input format, normalize findings to:
+
+```json
+{
+  "findings": [
+    {
+      "id": "TOB-ACME-1",
+      "title": "Missing access control in withdraw",
+      "severity": "High",
+      "difficulty": "Low",
+      "type": "Access Controls",
+      "files": ["contracts/Vault.sol"],
+      "description": "The withdraw function lacks...",
+      "recommendation": "Add onlyOwner modifier..."
+    }
+  ],
+  "metadata": {
+    "client": "ACME",
+    "date": "2024-01-15",
+    "format": "tob"
+  }
+}
+```
+
+This normalized format enables consistent processing regardless of source format.
+
+---
+
+## Handling Incomplete Reports
+
+When report lacks standard structure:
+
+### Missing Finding IDs
+
+Generate IDs based on:
+- Severity + sequence: `HIGH-1`, `HIGH-2`, `MEDIUM-1`
+- Position: `FINDING-1`, `FINDING-2`
+- File path: `VAULT-1`, `TOKEN-1`
+
+### Missing Severity
+
+Infer from:
+- Keywords: "critical", "severe", "important" → High
+- Impact description: "attacker can steal" → High
+- Default to "Undetermined" if unclear
+
+### Missing File References
+
+Search report for:
+- File paths: `/path/to/file`, `src/module/file.py`
+- Function names: `function()`, `method()`
+- Contract names: `Contract.function`
+
+---
+
+## Error Handling
+
+### File Not Found
+
+```
+Unable to read report at [path].
+Please verify the file exists and provide the correct path.
+```
+
+### Unsupported Format
+
+```
+Unable to parse report format.
+Supported formats: PDF, Markdown, JSON, HTML
+Please convert to a supported format or provide as Markdown.
+```
+
+### Empty Findings
+
+```
+No findings detected in the report.
+Please verify this is a security audit report with findings.
+If findings exist but weren't detected, provide them manually.
+```
+
+### Partial Parse
+
+```
+Parsed [N] findings, but some content may have been missed.
+Detected findings: [list IDs]
+Please verify all expected findings are included.
+```