@vigolium/piolium 0.0.1

package/skills/semgrep/references/scan-modes.md

@@ -0,0 +1,110 @@
+# Scan Modes Reference
+
+## Mode: Run All
+
+Full scan with all rulesets and severity levels. Current default behavior. No filtering applied — all findings are reported and triaged.
+
+## Mode: Important Only
+
+Focused on high-confidence security vulnerabilities. Excludes code quality, best practices, and low-confidence audit findings.
+
+### Pre-Filter: CLI Severity Flag
+
+Add these flags to every `semgrep` command:
+
+```bash
+--severity MEDIUM --severity HIGH --severity CRITICAL
+```
+
+This excludes LOW/INFO severity findings at scan time, reducing output volume before post-filtering.
+
+### Post-Filter: Metadata Criteria
+
+After scanning, filter each JSON result file to keep only findings matching ALL of:
+
+| Metadata Field | Accepted Values | Rationale |
+|---|---|---|
+| `extra.metadata.category` | `"security"` | Excludes correctness, best-practice, maintainability, performance |
+| `extra.metadata.confidence` | `"MEDIUM"`, `"HIGH"` | Excludes low-precision rules (high false positive rate) |
+| `extra.metadata.impact` | `"MEDIUM"`, `"HIGH"` | Excludes low-impact informational findings |
+
+**Third-party rules** (Trail of Bits, 0xdea, Decurity, etc.) may not have `confidence`/`impact`/`category` metadata. Findings **without** these metadata fields are **kept** — we cannot filter what is not annotated, and third-party rules are typically security-focused.
+
+### Semgrep Metadata Background
+
+Semgrep security rules have these metadata fields (required for `category: security` in the official registry):
+
+| Field | Purpose | Values |
+|---|---|---|
+| `severity` (top-level) | Overall rule severity, derived from likelihood × impact | `LOW`, `MEDIUM`, `HIGH`, `CRITICAL` |
+| `category` | Rule category | `security`, `correctness`, `best-practice`, `maintainability`, `performance` |
+| `confidence` | True positive rate of the rule (precision) | `LOW`, `MEDIUM`, `HIGH` |
+| `impact` | Potential damage if vulnerability is exploited | `LOW`, `MEDIUM`, `HIGH` |
+| `likelihood` | How likely the vulnerability is exploitable | `LOW`, `MEDIUM`, `HIGH` |
+| `subcategory` | Finding type | `vuln`, `audit`, `secure default` |
+
+Key relationship: `severity = f(likelihood, impact)` while `confidence` is independent (describes rule quality, not vulnerability severity).
+
+### Post-Filter jq Command
+
+Apply to each JSON result file after scanning:
+
+```bash
+# Filter a single result file
+jq '{
+  results: [.results[] |
+    ((.extra.metadata.category // "security") | ascii_downcase) as $cat |
+    ((.extra.metadata.confidence // "HIGH") | ascii_upcase) as $conf |
+    ((.extra.metadata.impact // "HIGH") | ascii_upcase) as $imp |
+    select(
+      ($cat == "security") and
+      ($conf == "MEDIUM" or $conf == "HIGH") and
+      ($imp == "MEDIUM" or $imp == "HIGH")
+    )
+  ],
+  errors: .errors,
+  paths: .paths
+}' "$f" > "${f%.json}-important.json"
+```
+
+Default values (`// "security"`, `// "HIGH"`) handle third-party rules without metadata — they pass all filters by default.
+
+### Filter All Result Files in a Directory
+
+Raw scan output lives in `$OUTPUT_DIR/raw/`. The filter creates `*-important.json` files alongside the originals — the raw files are preserved unmodified.
+
+```bash
+# Apply important-only filter to all scan result JSON files in raw/
+for f in "$OUTPUT_DIR/raw"/*-*.json; do
+  [[ "$f" == *-triage.json || "$f" == *-important.json ]] && continue
+  jq '{
+    results: [.results[] |
+      ((.extra.metadata.category // "security") | ascii_downcase) as $cat |
+      ((.extra.metadata.confidence // "HIGH") | ascii_upcase) as $conf |
+      ((.extra.metadata.impact // "HIGH") | ascii_upcase) as $imp |
+      select(
+        ($cat == "security") and
+        ($conf == "MEDIUM" or $conf == "HIGH") and
+        ($imp == "MEDIUM" or $imp == "HIGH")
+      )
+    ],
+    errors: .errors,
+    paths: .paths
+  }' "$f" > "${f%.json}-important.json"
+  BEFORE=$(jq '.results | length' "$f")
+  AFTER=$(jq '.results | length' "${f%.json}-important.json")
+  echo "$f: $BEFORE → $AFTER findings (filtered $(( BEFORE - AFTER )))"
+done
+```
+
+### Scanner Task Modifications
+
+In important-only mode, add `[SEVERITY_FLAGS]` to the scanner template:
+
+```bash
+semgrep [--pro if available] --metrics=off [SEVERITY_FLAGS] --config [RULESET] --json -o [OUTPUT_DIR]/raw/[lang]-[ruleset].json --sarif-output=[OUTPUT_DIR]/raw/[lang]-[ruleset].sarif [TARGET] &
+```
+
+Where `[SEVERITY_FLAGS]` is:
+- **Run all**: *(empty)*
+- **Important only**: `--severity MEDIUM --severity HIGH --severity CRITICAL`

package/skills/semgrep/references/scanner-task-prompt.md

@@ -0,0 +1,140 @@
+# Scanner Subagent Task Prompt
+
+Use this prompt template when spawning scanner Tasks in Step 4. Use `subagent_type: static-analysis:semgrep-scanner`.
+
+## Template
+
+```
+You are a Semgrep scanner for [LANGUAGE_CATEGORY].
+
+## Task
+Run Semgrep scans for [LANGUAGE] files and save results to [OUTPUT_DIR]/raw.
+
+## Pro Engine Status: [PRO_AVAILABLE: true/false]
+
+## Scan Mode: [SCAN_MODE: run-all/important-only]
+
+## APPROVED RULESETS (from user-confirmed plan)
+[LIST EXACT RULESETS USER APPROVED - DO NOT SUBSTITUTE]
+
+Example:
+- p/python
+- p/django
+- p/security-audit
+- p/secrets
+- https://github.com/trailofbits/semgrep-rules
+
+## Commands to Run (in parallel)
+
+### Clone GitHub URL rulesets first:
+```bash
+mkdir -p [OUTPUT_DIR]/repos
+# For each GitHub URL ruleset, clone into [OUTPUT_DIR]/repos/[name]:
+git clone --depth 1 https://github.com/org/repo [OUTPUT_DIR]/repos/repo-name
+```
+
+### Generate commands for EACH approved ruleset:
+```bash
+semgrep [--pro if available] --metrics=off [SEVERITY_FLAGS] [INCLUDE_FLAGS] --config [RULESET] --json -o [OUTPUT_DIR]/raw/[lang]-[ruleset].json --sarif-output=[OUTPUT_DIR]/raw/[lang]-[ruleset].sarif [TARGET] &
+```
+
+Wait for all to complete:
+```bash
+wait
+```
+
+### Clean up cloned repos:
+```bash
+[ -n "[OUTPUT_DIR]" ] && rm -rf [OUTPUT_DIR]/repos
+```
+
+## Critical Rules
+- Use ONLY the rulesets listed above - do not add or remove any
+- Always use --metrics=off (prevents sending telemetry to Semgrep servers)
+- Use --pro when Pro is available (enables cross-file taint tracking)
+- If scan mode is **important-only**, add `--severity MEDIUM --severity HIGH --severity CRITICAL` to every command
+- If scan mode is **run-all**, do NOT add severity flags
+- Run all rulesets in parallel with & and wait
+- For GitHub URL rulesets, always clone into [OUTPUT_DIR]/repos/ and use the local path as --config (do NOT pass URLs directly to semgrep — its URL handling is unreliable for repos with non-standard YAML)
+- Add `--include` flags for language-specific rulesets (e.g., `--include="*.py"` for p/python). Do NOT add `--include` to cross-language rulesets like p/security-audit, p/secrets, or third-party repos
+- After all scans complete, delete [OUTPUT_DIR]/repos/ to avoid leaving cloned repos behind
+
+## Output
+Report:
+- Number of findings per ruleset
+- Any scan errors
+- File paths of JSON results (in [OUTPUT_DIR]/raw/)
+- [If Pro] Note any cross-file findings detected
+```
+
+## Variable Substitutions
+
+| Variable | Description | Example |
+|----------|-------------|---------|
+| `[LANGUAGE_CATEGORY]` | Language group being scanned | Python, JavaScript, Docker |
+| `[LANGUAGE]` | Specific language | Python, TypeScript, Go |
+| `[OUTPUT_DIR]` | Output directory (absolute path, resolved in Step 1) | /path/to/static_analysis_semgrep_1 |
+| `[PRO_AVAILABLE]` | Whether Pro engine is available | true, false |
+| `[SEVERITY_FLAGS]` | Severity pre-filter flags | *(empty)* for run-all, `--severity MEDIUM --severity HIGH --severity CRITICAL` for important-only |
+| `[INCLUDE_FLAGS]` | File extension filter for language-specific rulesets | `--include="*.py"` for Python rulesets, *(empty)* for cross-language rulesets like p/security-audit, p/secrets, or third-party repos |
+| `[RULESET]` | Semgrep ruleset identifier or local clone path | p/python, [OUTPUT_DIR]/repos/semgrep-rules |
+| `[TARGET]` | Absolute path to directory to scan | /path/to/codebase |
+
+## Example: Python Scanner Task
+
+```
+You are a Semgrep scanner for Python.
+
+## Task
+Run Semgrep scans for Python files and save results to /path/to/static_analysis_semgrep_1/raw.
+
+## Pro Engine Status: true
+
+## Scan Mode: run-all
+
+## APPROVED RULESETS (from user-confirmed plan)
+- p/python
+- p/django
+- p/security-audit
+- p/secrets
+- https://github.com/trailofbits/semgrep-rules
+
+## Commands to Run (in parallel)
+
+### Clone GitHub URL rulesets first:
+```bash
+mkdir -p /path/to/static_analysis_semgrep_1/repos
+git clone --depth 1 https://github.com/trailofbits/semgrep-rules /path/to/static_analysis_semgrep_1/repos/trailofbits
+```
+
+### Run scans:
+```bash
+semgrep --pro --metrics=off --include="*.py" --config p/python --json -o /path/to/static_analysis_semgrep_1/raw/python-python.json --sarif-output=/path/to/static_analysis_semgrep_1/raw/python-python.sarif /path/to/codebase &
+semgrep --pro --metrics=off --include="*.py" --config p/django --json -o /path/to/static_analysis_semgrep_1/raw/python-django.json --sarif-output=/path/to/static_analysis_semgrep_1/raw/python-django.sarif /path/to/codebase &
+semgrep --pro --metrics=off --config p/security-audit --json -o /path/to/static_analysis_semgrep_1/raw/python-security-audit.json --sarif-output=/path/to/static_analysis_semgrep_1/raw/python-security-audit.sarif /path/to/codebase &
+semgrep --pro --metrics=off --config p/secrets --json -o /path/to/static_analysis_semgrep_1/raw/python-secrets.json --sarif-output=/path/to/static_analysis_semgrep_1/raw/python-secrets.sarif /path/to/codebase &
+semgrep --pro --metrics=off --config /path/to/static_analysis_semgrep_1/repos/trailofbits --json -o /path/to/static_analysis_semgrep_1/raw/python-trailofbits.json --sarif-output=/path/to/static_analysis_semgrep_1/raw/python-trailofbits.sarif /path/to/codebase &
+wait
+```
+
+### Clean up cloned repos:
+```bash
+rm -rf /path/to/static_analysis_semgrep_1/repos
+```
+
+## Critical Rules
+- Use ONLY the rulesets listed above - do not add or remove any
+- Always use --metrics=off
+- Use --pro when Pro is available
+- Run all rulesets in parallel with & and wait
+- Clone GitHub URL rulesets into the output dir repos/ subfolder, use local path as --config
+- Add --include="*.py" to language-specific rulesets (p/python, p/django) but NOT to p/security-audit, p/secrets, or third-party repos
+- Delete repos/ after scanning
+
+## Output
+Report:
+- Number of findings per ruleset
+- Any scan errors
+- File paths of JSON results (in raw/ subdirectory)
+- Note any cross-file findings detected
+```

package/skills/semgrep/scripts/merge_sarif.py

@@ -0,0 +1,203 @@
+# /// script
+# requires-python = ">=3.11"
+# dependencies = []
+# ///
+"""Merge SARIF files into a single consolidated output.
+
+Usage:
+    uv run merge_sarif.py RAW_DIR OUTPUT_FILE
+
+Reads *.sarif files from RAW_DIR (e.g., $OUTPUT_DIR/raw), produces
+OUTPUT_FILE (e.g., $OUTPUT_DIR/results/results.sarif) containing all
+findings merged and deduplicated.
+
+Attempts to use SARIF Multitool for merging if available, falls back to
+pure Python implementation.
+"""
+
+from __future__ import annotations
+
+import json
+import shutil
+import subprocess
+import sys
+import tempfile
+from pathlib import Path
+
+
+def has_sarif_multitool() -> bool:
+    """Check if SARIF Multitool is pre-installed via npx."""
+    if not shutil.which("npx"):
+        return False
+    try:
+        result = subprocess.run(
+            ["npx", "--no-install", "@microsoft/sarif-multitool", "--version"],
+            capture_output=True,
+            timeout=30,
+        )
+        return result.returncode == 0
+    except subprocess.TimeoutExpired:
+        print("Warning: SARIF Multitool version check timed out", file=sys.stderr)
+        return False
+    except FileNotFoundError:
+        return False
+    except OSError as e:
+        print(f"Warning: Failed to check SARIF Multitool: {e}", file=sys.stderr)
+        return False
+
+
+def merge_with_multitool(sarif_files: list[Path]) -> dict | None:
+    """Use SARIF Multitool to merge SARIF files. Returns merged SARIF or None."""
+    if not sarif_files:
+        return None
+
+    with tempfile.NamedTemporaryFile(suffix=".sarif", delete=False) as tmp:
+        tmp_path = Path(tmp.name)
+
+    try:
+        cmd = [
+            "npx",
+            "--no-install",
+            "@microsoft/sarif-multitool",
+            "merge",
+            *[str(f) for f in sarif_files],
+            "--output-file",
+            str(tmp_path),
+            "--force",
+        ]
+        result = subprocess.run(cmd, capture_output=True, timeout=120)
+        if result.returncode != 0:
+            print(f"SARIF Multitool merge failed: {result.stderr.decode()}", file=sys.stderr)
+            return None
+
+        return json.loads(tmp_path.read_text())
+    except subprocess.TimeoutExpired as e:
+        print(f"SARIF Multitool timed out: {e}", file=sys.stderr)
+        return None
+    except json.JSONDecodeError as e:
+        print(f"SARIF Multitool produced invalid JSON: {e}", file=sys.stderr)
+        return None
+    except FileNotFoundError as e:
+        print(f"SARIF Multitool not found: {e}", file=sys.stderr)
+        return None
+    except OSError as e:
+        print(f"SARIF Multitool OS error ({type(e).__name__}): {e}", file=sys.stderr)
+        return None
+    finally:
+        tmp_path.unlink(missing_ok=True)
+
+
+def merge_sarif_pure_python(sarif_files: list[Path]) -> dict:
+    """Pure Python SARIF merge (fallback)."""
+    merged = {
+        "version": "2.1.0",
+        "$schema": "https://json.schemastore.org/sarif-2.1.0.json",
+        "runs": [],
+    }
+
+    seen_rules: dict[str, dict] = {}
+    all_results: list[dict] = []
+    seen_results: set[tuple[str, str, int]] = set()
+    tool_info: dict | None = None
+    skipped_files: list[str] = []
+
+    for sarif_file in sorted(sarif_files):
+        try:
+            data = json.loads(sarif_file.read_text())
+        except json.JSONDecodeError as e:
+            print(f"Warning: Failed to parse {sarif_file}: {e}", file=sys.stderr)
+            skipped_files.append(str(sarif_file))
+            continue
+
+        for run in data.get("runs", []):
+            if tool_info is None and run.get("tool"):
+                tool_info = run["tool"]
+
+            driver = run.get("tool", {}).get("driver", {})
+            for rule in driver.get("rules", []):
+                rule_id = rule.get("id", "")
+                if rule_id and rule_id not in seen_rules:
+                    seen_rules[rule_id] = rule
+
+            for result in run.get("results", []):
+                rule_id = result.get("ruleId", "")
+                uri = ""
+                start_line = 0
+                locations = result.get("locations", [])
+                if locations:
+                    phys = locations[0].get("physicalLocation", {})
+                    uri = phys.get("artifactLocation", {}).get("uri", "")
+                    start_line = phys.get("region", {}).get("startLine", 0)
+                dedup_key = (rule_id, uri, start_line)
+                if dedup_key in seen_results:
+                    continue
+                seen_results.add(dedup_key)
+                all_results.append(result)
+
+    if all_results:
+        merged_run = {
+            "tool": tool_info or {"driver": {"name": "semgrep", "rules": []}},
+            "results": all_results,
+        }
+        merged_run["tool"]["driver"]["rules"] = list(seen_rules.values())
+        merged["runs"].append(merged_run)
+
+    if skipped_files:
+        print(
+            f"WARNING: {len(skipped_files)} of {len(sarif_files)} SARIF files "
+            f"could not be parsed. Results may be incomplete.",
+            file=sys.stderr,
+        )
+        for sf in skipped_files:
+            print(f"  Skipped: {sf}", file=sys.stderr)
+
+    return merged
+
+
+def main() -> int:
+    if len(sys.argv) != 3:
+        print(f"Usage: {sys.argv[0]} RAW_DIR OUTPUT_FILE", file=sys.stderr)
+        return 1
+
+    raw_dir = Path(sys.argv[1])
+    output_file = Path(sys.argv[2])
+
+    if not raw_dir.is_dir():
+        print(f"Error: {raw_dir} is not a directory", file=sys.stderr)
+        return 1
+
+    # Collect SARIF files from raw directory only
+    sarif_files = sorted(raw_dir.glob("*.sarif"))
+    print(f"Found {len(sarif_files)} SARIF files to merge in {raw_dir}")
+
+    if not sarif_files:
+        print("No SARIF files found, nothing to merge", file=sys.stderr)
+        return 1
+
+    # Ensure output directory exists
+    output_file.parent.mkdir(parents=True, exist_ok=True)
+
+    # Try SARIF Multitool first, fall back to pure Python
+    merged: dict | None = None
+    if has_sarif_multitool():
+        print("Using SARIF Multitool for merge...")
+        merged = merge_with_multitool(sarif_files)
+        if merged:
+            print("SARIF Multitool merge successful")
+
+    if merged is None:
+        print("Using pure Python merge (SARIF Multitool not available or failed)")
+        merged = merge_sarif_pure_python(sarif_files)
+
+    result_count = sum(len(run.get("results", [])) for run in merged.get("runs", []))
+    print(f"Merged SARIF contains {result_count} findings")
+
+    # Write output
+    output_file.write_text(json.dumps(merged, indent=2))
+    print(f"Written to {output_file}")
+
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())