rtexit-method 0.1.0 → 0.1.1

package/packaged-assets/.agents/skills/rt-osint/SKILL.md

@@ -0,0 +1,775 @@
+---
+name: rt-osint
+description: "OSINT reconnaissance skill. Use when gathering intelligence on a target — email harvesting, subdomain discovery, employee enumeration, social media footprint, credential leaks, company structure. Covers passive OSINT from beginner (Google Dorks) to expert (custom correlation frameworks). References: OSINT Framework, IntelTechniques, theHarvester, Amass, Shodan, DeHashed."
+---
+
+# rt-osint — Red Team OSINT Reconnaissance Skill
+
+## Overview
+
+This skill covers passive and semi-passive Open Source Intelligence (OSINT) collection against a target organization. The goal is to build a detailed intelligence picture before any active engagement begins — identifying attack surface, personnel, infrastructure, leaked credentials, and technology stack without triggering detection.
+
+OSINT is the first phase of every professional Red Team engagement. Intelligence gathered here directly informs phishing lures, password spray lists, subdomain targeting, and social engineering pretexts.
+
+Scope of this skill:
+- Email address harvesting and validation
+- Subdomain and DNS enumeration (passive)
+- Employee enumeration via LinkedIn, GitHub, and job boards
+- Social media footprint mapping
+- Credential leak discovery (DeHashed, HaveIBeenPwned, breach dumps)
+- Company structure, M&A history, subsidiary mapping
+- Technology stack fingerprinting (Shodan, Censys, BuiltWith)
+- ASN and IP range discovery
+- Document metadata extraction
+
+> All commands assume Kali Linux unless noted. Replace `TARGET` and `DOMAIN` with the actual engagement target.
+
+---
+
+## Skill Levels
+
+### BEGINNER — Passive Recon with Public Tools
+
+At this level you use no accounts, no APIs, and no authentication. All data comes from search engines and publicly accessible web pages.
+
+**Google Dorks**
+
+```bash
+# Find email addresses on the target domain
+site:TARGET.com "@TARGET.com"
+
+# Find subdomains via Google
+site:*.TARGET.com -www
+
+# Find exposed documents
+site:TARGET.com filetype:pdf OR filetype:docx OR filetype:xlsx
+
+# Find login portals
+site:TARGET.com inurl:login OR inurl:portal OR inurl:admin
+
+# Find employee names on LinkedIn
+site:linkedin.com/in "TARGET Company" "Current"
+
+# Find exposed configuration files
+site:TARGET.com ext:env OR ext:cfg OR ext:conf OR ext:ini
+
+# Find internal paths in cached pages
+cache:TARGET.com
+```
+
+**Whois and DNS lookups**
+
+```bash
+# Whois lookup
+whois TARGET.com
+
+# Basic DNS enumeration
+dig TARGET.com ANY
+dig TARGET.com MX
+dig TARGET.com TXT
+dig TARGET.com NS
+host -t ns TARGET.com
+host -t mx TARGET.com
+
+# Reverse DNS on the main IP
+host $(dig +short TARGET.com)
+
+# Check SPF, DKIM, DMARC records (email security posture)
+dig TARGET.com TXT | grep -i spf
+dig _dmarc.TARGET.com TXT
+```
+
+**Certificate Transparency Logs**
+
+```bash
+# Query crt.sh for all known subdomains
+curl -s "https://crt.sh/?q=%.TARGET.com&output=json" | \
+  python3 -c "import sys,json; [print(c['name_value']) for c in json.load(sys.stdin)]" | \
+  sort -u
+
+# Save output
+curl -s "https://crt.sh/?q=%.TARGET.com&output=json" | \
+  python3 -c "import sys,json; [print(c['name_value']) for c in json.load(sys.stdin)]" | \
+  sort -u > output/subdomains-crt.txt
+```
+
+---
+
+### INTERMEDIATE — API-Driven Enumeration
+
+At this level you use tool suites with API keys, producing structured output suitable for downstream analysis.
+
+**theHarvester**
+
+```bash
+# Install
+pip3 install theHarvester
+
+# Harvest emails, subdomains, hosts from multiple sources
+theHarvester -d TARGET.com -b all -l 500 -f output/theharvester-TARGET
+
+# Individual sources
+theHarvester -d TARGET.com -b google
+theHarvester -d TARGET.com -b bing
+theHarvester -d TARGET.com -b linkedin
+theHarvester -d TARGET.com -b shodan
+theHarvester -d TARGET.com -b hunter
+theHarvester -d TARGET.com -b github-code
+```
+
+**Amass — Passive Subdomain Enumeration**
+
+```bash
+# Passive only (safe for pre-authorization recon)
+amass enum -passive -d TARGET.com -o output/amass-passive.txt
+
+# With API keys configured in ~/.config/amass/datasources.yaml
+amass enum -passive -d TARGET.com -config ~/.config/amass/datasources.yaml -o output/amass-passive.txt
+
+# Show discovered assets with sources
+amass enum -passive -d TARGET.com -v 2>&1 | tee output/amass-verbose.txt
+
+# Database query after collection
+amass db -d TARGET.com -show
+```
+
+**Subfinder**
+
+```bash
+# Install
+go install -v github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latest
+
+# Passive subdomain discovery
+subfinder -d TARGET.com -o output/subfinder.txt -all -recursive
+
+# With silent output for piping
+subfinder -d TARGET.com -silent | tee output/subfinder-silent.txt
+
+# Resolve discovered subdomains
+subfinder -d TARGET.com -silent | httpx -silent -o output/live-subdomains.txt
+```
+
+**Shodan**
+
+```bash
+# Install CLI
+pip3 install shodan
+shodan init YOUR_API_KEY
+
+# Search by organization name
+shodan search "org:\"Target Company Name\""
+
+# Search by domain
+shodan search "hostname:TARGET.com"
+
+# Download full results
+shodan download output/shodan-TARGET hostname:TARGET.com
+shodan parse --fields ip_str,port,hostnames,org output/shodan-TARGET.json.gz > output/shodan-parsed.csv
+
+# Find open RDP, SMB, VNC
+shodan search "org:\"Target Company\" port:3389"
+shodan search "org:\"Target Company\" port:445"
+
+# Find exposed Elasticsearch
+shodan search "org:\"Target Company\" product:Elastic"
+
+# IP info
+shodan host TARGET_IP
+```
+
+**Hunter.io Email Harvesting**
+
+```bash
+# Via API
+curl "https://api.hunter.io/v2/domain-search?domain=TARGET.com&api_key=YOUR_KEY" \
+  -o output/hunter-TARGET.json
+
+# Parse emails
+cat output/hunter-TARGET.json | python3 -c "
+import sys, json
+data = json.load(sys.stdin)
+for e in data['data']['emails']:
+    print(e['value'], e.get('first_name',''), e.get('last_name',''), e.get('position',''))
+" > output/hunter-emails.txt
+```
+
+---
+
+### ADVANCED — Deep Enumeration and Correlation
+
+At this level you correlate data across multiple sources, enumerate infrastructure beyond the primary domain, and build employee target lists.
+
+**ASN and IP Range Discovery**
+
+```bash
+# Find ASN for organization
+curl -s "https://api.bgpview.io/search?query_term=Target+Company" | \
+  python3 -c "import sys,json; d=json.load(sys.stdin); [print(a['asn'],a['description']) for a in d['data']['asns']]"
+
+# Get all prefixes for an ASN
+ASN=12345
+curl -s "https://api.bgpview.io/asn/${ASN}/prefixes" | \
+  python3 -c "import sys,json; d=json.load(sys.stdin); [print(p['prefix']) for p in d['data']['ipv4_prefixes']]" \
+  > output/asn-prefixes.txt
+
+# Reverse WHOIS — find all domains registered by same org
+curl "https://www.whoisxmlapi.com/whoisserver/WhoisService?apiKey=YOUR_KEY&domainName=TARGET.com&outputFormat=JSON"
+```
+
+**LinkedIn Employee Enumeration**
+
+```bash
+# Install linkedin2username
+git clone https://github.com/initstring/linkedin2username
+cd linkedin2username
+pip3 install -r requirements.txt
+
+# Enumerate employees (requires LinkedIn account)
+python3 linkedin2username.py -u YOUR_LINKEDIN_EMAIL -c "Target Company" \
+  -o output/linkedin-employees.txt -s 1
+
+# Generate username formats for password spraying
+# linkedin2username produces: firstname.lastname, f.lastname, firstnamel, etc.
+```
+
+**GitHub Recon**
+
+```bash
+# Install gitrob or trufflehog
+pip3 install trufflehog
+
+# Search GitHub for secrets in target org repos
+trufflehog github --org=TARGET_ORG_NAME --only-verified \
+  --json > output/trufflehog-TARGET.json
+
+# Manual GitHub search dorks
+# "TARGET.com" password
+# "TARGET.com" secret
+# "TARGET.com" api_key
+# org:TARGET_ORG filename:.env
+# org:TARGET_ORG filename:id_rsa
+
+# gitleaks for local repo scanning
+gitleaks detect --source /path/to/cloned/repo --report-path output/gitleaks-report.json
+```
+
+**Credential Leak Discovery**
+
+```bash
+# DeHashed API (paid)
+curl -H "Accept: application/json" \
+  "https://api.dehashed.com/search?query=domain:TARGET.com&size=100" \
+  -u "YOUR_EMAIL:YOUR_API_KEY" | python3 -m json.tool > output/dehashed-TARGET.json
+
+# Parse leaked passwords
+cat output/dehashed-TARGET.json | python3 -c "
+import sys, json
+data = json.load(sys.stdin)
+for entry in data.get('entries', []):
+    print(entry.get('email',''), entry.get('password',''), entry.get('hashed_password',''))
+" > output/leaked-creds.txt
+
+# HaveIBeenPwned — check domain breach exposure
+curl -H "hibp-api-key: YOUR_KEY" \
+  "https://haveibeenpwned.com/api/v3/breachesforaccount/email@TARGET.com" \
+  > output/hibp-result.json
+
+# h8mail — email breach lookup
+pip3 install h8mail
+h8mail -t TARGET.com -bc output/h8mail-breach.txt
+```
+
+**Document Metadata Extraction**
+
+```bash
+# Download all PDFs from target domain
+wget -r -l2 -A "*.pdf,*.docx,*.xlsx,*.pptx" --no-parent -nd \
+  -P output/docs/ https://TARGET.com/
+
+# Extract metadata with ExifTool
+exiftool output/docs/*.pdf | grep -E "Author|Creator|Company|Producer|Last Modified" \
+  > output/pdf-metadata.txt
+
+# Extract with metagoofil
+metagoofil -d TARGET.com -t pdf,doc,xls,ppt -l 50 -n 10 -o output/metagoofil/
+```
+
+**Technology Fingerprinting**
+
+```bash
+# WhatWeb
+whatweb -a 3 TARGET.com -v | tee output/whatweb.txt
+
+# BuiltWith API
+curl "https://api.builtwith.com/v20/api.json?KEY=YOUR_KEY&LOOKUP=TARGET.com" \
+  > output/builtwith.json
+
+# Wappalyzer CLI
+npm install -g wappalyzer
+wappalyzer https://TARGET.com --pretty > output/wappalyzer.json
+
+# httpx — banner grab and tech detection on all live subdomains
+cat output/live-subdomains.txt | httpx -tech-detect -status-code -title \
+  -o output/httpx-tech.txt -json
+```
+
+---
+
+### EXPERT — Custom Correlation Frameworks
+
+At this level you build automated correlation pipelines, integrate multiple data sources, and produce actionable intelligence packages.
+
+**Automated OSINT Pipeline Script**
+
+```bash
+#!/bin/bash
+# rt-osint-pipeline.sh
+# Full OSINT collection pipeline for a target domain
+# Usage: ./rt-osint-pipeline.sh TARGET.com "Company Name" ENGAGEMENT_ID
+
+DOMAIN=$1
+COMPANY=$2
+ENG_ID=$3
+OUTDIR="engagements/${ENG_ID}/osint"
+
+mkdir -p "${OUTDIR}"/{subdomains,emails,employees,infra,leaks,docs,social}
+
+echo "[*] Starting OSINT pipeline for ${DOMAIN} (${COMPANY}) — $(date)"
+
+# Phase 1: Subdomain discovery
+echo "[*] Phase 1: Subdomain discovery"
+subfinder -d "${DOMAIN}" -silent -all -recursive > "${OUTDIR}/subdomains/subfinder.txt"
+amass enum -passive -d "${DOMAIN}" -o "${OUTDIR}/subdomains/amass.txt" 2>/dev/null
+curl -s "https://crt.sh/?q=%.${DOMAIN}&output=json" | \
+  python3 -c "import sys,json; [print(c['name_value']) for c in json.load(sys.stdin)]" \
+  2>/dev/null | sort -u > "${OUTDIR}/subdomains/crtsh.txt"
+
+# Merge and deduplicate
+cat "${OUTDIR}/subdomains/"*.txt | sort -u > "${OUTDIR}/subdomains/all-subdomains.txt"
+echo "[+] Found $(wc -l < "${OUTDIR}/subdomains/all-subdomains.txt") unique subdomains"
+
+# Phase 2: Resolve live hosts
+echo "[*] Phase 2: Resolving live hosts"
+cat "${OUTDIR}/subdomains/all-subdomains.txt" | \
+  httpx -silent -status-code -title -tech-detect -json \
+  -o "${OUTDIR}/subdomains/live-hosts.json"
+
+# Phase 3: Email harvesting
+echo "[*] Phase 3: Email harvesting"
+theHarvester -d "${DOMAIN}" -b google,bing,hunter,linkedin -l 500 \
+  -f "${OUTDIR}/emails/theharvester" 2>/dev/null
+
+# Phase 4: Infrastructure mapping
+echo "[*] Phase 4: Infrastructure mapping"
+dig "${DOMAIN}" ANY > "${OUTDIR}/infra/dns-any.txt"
+whois "${DOMAIN}" > "${OUTDIR}/infra/whois.txt"
+
+echo "[+] Pipeline complete. Output: ${OUTDIR}"
+```
+
+**Python Correlation Script — Build Employee Target List**
+
+```python
+#!/usr/bin/env python3
+# correlate_employees.py
+# Correlates LinkedIn names with email format to build spray list
+
+import sys
+import re
+
+def generate_formats(first, last, domain):
+    f = first.lower().strip()
+    l = last.lower().strip()
+    return [
+        f"{f}.{l}@{domain}",
+        f"{f[0]}{l}@{domain}",
+        f"{f}{l[0]}@{domain}",
+        f"{f}_{l}@{domain}",
+        f"{f}@{domain}",
+        f"{l}{f[0]}@{domain}",
+        f"{f[0]}.{l}@{domain}",
+    ]
+
+if __name__ == "__main__":
+    domain = sys.argv[1]
+    names_file = sys.argv[2]
+
+    with open(names_file) as f:
+        for line in f:
+            parts = line.strip().split()
+            if len(parts) >= 2:
+                first, last = parts[0], parts[-1]
+                for email in generate_formats(first, last, domain):
+                    print(email)
+```
+
+```bash
+# Usage
+python3 correlate_employees.py TARGET.com output/linkedin-employees.txt | sort -u \
+  > output/candidate-emails.txt
+
+# Validate emails with verify-email or emailhippo
+cat output/candidate-emails.txt | emailhippo -o output/validated-emails.txt
+```
+
+**SpiderFoot — Automated OSINT Framework**
+
+```bash
+# Install
+git clone https://github.com/smicallef/spiderfoot
+cd spiderfoot
+pip3 install -r requirements.txt
+
+# Launch web UI
+python3 sf.py -l 127.0.0.1:5001
+
+# CLI scan — passive modules only
+python3 sfcli.py -s TARGET.com -t INTERNET_NAME -m sfp_dnsresolve,sfp_ssl,sfp_crt,\
+sfp_hunter,sfp_shodan,sfp_passivetotal,sfp_circl,sfp_threatcrowd \
+-o output/spiderfoot-TARGET.json
+```
+
+**Maltego Integration**
+
+```bash
+# Export structured data for Maltego import
+# Convert theHarvester XML output
+python3 - << 'EOF'
+import xml.etree.ElementTree as ET
+import json
+
+tree = ET.parse('output/emails/theharvester.xml')
+root = tree.getroot()
+
+entities = []
+for email in root.findall('.//email'):
+    entities.append({"type": "Email", "value": email.text})
+for host in root.findall('.//host'):
+    entities.append({"type": "Host", "value": host.text})
+
+with open('output/maltego-import.json', 'w') as f:
+    json.dump(entities, f, indent=2)
+
+print(f"Exported {len(entities)} entities for Maltego")
+EOF
+```
+
+---
+
+## Step-by-Step Workflow
+
+1. **Create engagement directory structure**
+
+```bash
+ENG_ID="ENG-2024-001"
+DOMAIN="target.com"
+mkdir -p engagements/${ENG_ID}/osint/{subdomains,emails,employees,infra,leaks,docs,social,screenshots}
+cd engagements/${ENG_ID}/osint
+```
+
+2. **Run passive DNS and certificate recon**
+
+```bash
+dig ${DOMAIN} ANY > infra/dns-any.txt
+whois ${DOMAIN} > infra/whois.txt
+curl -s "https://crt.sh/?q=%.${DOMAIN}&output=json" | \
+  python3 -c "import sys,json; [print(c['name_value']) for c in json.load(sys.stdin)]" | \
+  sort -u > subdomains/crtsh.txt
+```
+
+3. **Run subfinder and amass in parallel**
+
+```bash
+subfinder -d ${DOMAIN} -silent -all -recursive -o subdomains/subfinder.txt &
+amass enum -passive -d ${DOMAIN} -o subdomains/amass.txt &
+wait
+cat subdomains/*.txt | sort -u > subdomains/all-subdomains.txt
+```
+
+4. **Resolve live hosts and fingerprint technologies**
+
+```bash
+cat subdomains/all-subdomains.txt | \
+  httpx -silent -status-code -title -tech-detect -json -o subdomains/live-hosts.json
+cat subdomains/live-hosts.json | python3 -c "
+import sys, json
+for line in sys.stdin:
+    h = json.loads(line)
+    print(h.get('url',''), h.get('status_code',''), h.get('title',''))
+" > subdomains/live-hosts-summary.txt
+```
+
+5. **Harvest email addresses**
+
+```bash
+theHarvester -d ${DOMAIN} -b all -l 500 -f emails/theharvester
+cat emails/theharvester.json | python3 -c "
+import sys, json
+d = json.load(sys.stdin)
+for e in d.get('emails', []):
+    print(e)
+" | sort -u > emails/all-emails.txt
+```
+
+6. **Enumerate employees from LinkedIn**
+
+```bash
+cd linkedin2username
+python3 linkedin2username.py -u YOUR_LINKEDIN -c "Target Company Name" \
+  -o ../employees/linkedin-names.txt -s 1
+cd ..
+python3 /opt/rt-tools/correlate_employees.py ${DOMAIN} employees/linkedin-names.txt \
+  > employees/candidate-emails.txt
+```
+
+7. **Check for credential leaks**
+
+```bash
+curl -H "Accept: application/json" \
+  "https://api.dehashed.com/search?query=domain:${DOMAIN}&size=200" \
+  -u "EMAIL:API_KEY" | python3 -m json.tool > leaks/dehashed.json
+
+cat leaks/dehashed.json | python3 -c "
+import sys, json
+data = json.load(sys.stdin)
+for e in data.get('entries', []):
+    pw = e.get('password') or e.get('hashed_password','')
+    print(f\"{e.get('email','')}\t{pw}\")
+" > leaks/leaked-creds.txt
+
+echo "[+] Found $(wc -l < leaks/leaked-creds.txt) leaked credential entries"
+```
+
+8. **Shodan infrastructure scan**
+
+```bash
+shodan search "org:\"Target Company\"" --fields ip_str,port,hostnames,os,product \
+  > infra/shodan-results.txt
+shodan search "org:\"Target Company\" has_vuln:true" --fields ip_str,port,vulns \
+  > infra/shodan-vulns.txt
+```
+
+9. **Screenshot all live web services**
+
+```bash
+cat subdomains/live-hosts.json | python3 -c "
+import sys, json
+for line in sys.stdin:
+    print(json.loads(line).get('url',''))
+" | aquatone -out screenshots/ -threads 5 -timeout 3000
+```
+
+10. **Extract document metadata**
+
+```bash
+metagoofil -d ${DOMAIN} -t pdf,doc,xls,ppt,docx,xlsx,pptx \
+  -l 50 -n 10 -o docs/metagoofil/
+exiftool docs/metagoofil/*.pdf 2>/dev/null | \
+  grep -E "Author|Creator|Company|Software" | sort -u > docs/metadata-authors.txt
+```
+
+11. **Generate final intelligence summary**
+
+```bash
+python3 /opt/rt-tools/osint-summary.py \
+  --domain ${DOMAIN} \
+  --subdomains subdomains/all-subdomains.txt \
+  --live subdomains/live-hosts-summary.txt \
+  --emails emails/all-emails.txt \
+  --employees employees/candidate-emails.txt \
+  --leaks leaks/leaked-creds.txt \
+  --output ../reports/osint-summary.md
+```
+
+12. **Feed into RTExit autodoc engine**
+
+```bash
+# RTExit autodoc integration
+rtx-doc ingest \
+  --engagement ${ENG_ID} \
+  --phase osint \
+  --artifacts ./engagements/${ENG_ID}/osint \
+  --tag "passive-recon"
+
+# Log OSINT completion event
+rtx-doc event \
+  --engagement ${ENG_ID} \
+  --type phase-complete \
+  --phase osint \
+  --notes "$(wc -l < subdomains/all-subdomains.txt) subdomains, $(wc -l < emails/all-emails.txt) emails, $(wc -l < leaks/leaked-creds.txt) leaked credentials"
+```
+
+---
+
+## Tools Reference
+
+| Tool | Purpose | URL |
+|---|---|---|
+| theHarvester | Email, subdomain, and host harvesting | https://github.com/laramies/theHarvester |
+| Amass | Subdomain enumeration and mapping | https://github.com/owasp-amass/amass |
+| Subfinder | Passive subdomain discovery | https://github.com/projectdiscovery/subfinder |
+| httpx | HTTP probing and tech detection | https://github.com/projectdiscovery/httpx |
+| Shodan CLI | Internet-facing asset discovery | https://github.com/achillean/shodan-python |
+| SpiderFoot | Automated multi-source OSINT | https://github.com/smicallef/spiderfoot |
+| linkedin2username | LinkedIn employee enumeration | https://github.com/initstring/linkedin2username |
+| TruffleHog | GitHub secret scanning | https://github.com/trufflesecurity/trufflehog |
+| gitleaks | Git repository secret scanning | https://github.com/gitleaks/gitleaks |
+| Aquatone | Web screenshot utility | https://github.com/michenriksen/aquatone |
+| metagoofil | Document metadata extraction | https://github.com/laramies/metagoofil |
+| ExifTool | Metadata reader/writer | https://github.com/exiftool/exiftool |
+| h8mail | Email breach lookup | https://github.com/khast3x/h8mail |
+| Maltego | Visual link analysis | https://www.maltego.com |
+| WhatWeb | Web technology fingerprinting | https://github.com/urbanadventurer/WhatWeb |
+| SecLists | Wordlists for all phases | https://github.com/danielmiessler/SecLists |
+
+**SecLists paths used in this skill:**
+
+```
+/usr/share/seclists/Discovery/DNS/subdomains-top1million-110000.txt
+/usr/share/seclists/Discovery/DNS/fierce-hostlist.txt
+/usr/share/seclists/Passwords/Common-Credentials/10-million-password-list-top-10000.txt
+/usr/share/seclists/Usernames/Names/names.txt
+```
+
+---
+
+## Output Instructions
+
+### File Organization
+
+```
+engagements/
+  ENG-YYYY-NNN/
+    osint/
+      subdomains/
+        crtsh.txt              # Certificate transparency subdomains
+        subfinder.txt          # Subfinder results
+        amass.txt              # Amass results
+        all-subdomains.txt     # Merged, deduplicated
+        live-hosts.json        # httpx JSON output
+        live-hosts-summary.txt # Human-readable live host summary
+      emails/
+        theharvester.json      # Raw theHarvester output
+        all-emails.txt         # Merged, deduplicated
+      employees/
+        linkedin-names.txt     # Raw names from LinkedIn
+        candidate-emails.txt   # Generated email format candidates
+        validated-emails.txt   # Validated live email addresses
+      infra/
+        dns-any.txt            # Full DNS record dump
+        whois.txt              # WHOIS data
+        shodan-results.txt     # Shodan host data
+        shodan-vulns.txt       # Shodan vulnerability findings
+        asn-prefixes.txt       # ASN IP ranges
+      leaks/
+        dehashed.json          # DeHashed API response
+        leaked-creds.txt       # Parsed email:password pairs
+        hibp-results.json      # HaveIBeenPwned results
+      docs/
+        metagoofil/            # Downloaded documents
+        metadata-authors.txt   # Extracted author/company metadata
+      social/
+        github-findings.json   # TruffleHog/gitleaks findings
+      screenshots/             # Aquatone web screenshots
+    reports/
+      osint-summary.md         # Executive OSINT summary
+```
+
+### Naming Convention
+
+- All output files use lowercase with hyphens
+- Include date suffix for repeated runs: `all-subdomains-2024-01-15.txt`
+- Raw API responses kept in original format (JSON/XML) for audit trail
+- Parsed/processed output saved separately from raw
+
+### RTExit Autodoc Integration
+
+After each phase, ingest artifacts into the RTExit autodoc engine:
+
+```bash
+# Ingest all OSINT artifacts
+rtx-doc ingest --engagement ENG_ID --phase osint --artifacts ./engagements/ENG_ID/osint
+
+# Tag high-value findings
+rtx-doc finding \
+  --engagement ENG_ID \
+  --title "Leaked credentials discovered for domain" \
+  --severity high \
+  --evidence leaks/leaked-creds.txt \
+  --recommendation "Initiate password reset enforcement and notify SOC"
+
+# Generate phase report
+rtx-doc report \
+  --engagement ENG_ID \
+  --phase osint \
+  --template osint-standard \
+  --output reports/osint-report.pdf
+```
+
+---
+
+## Resources
+
+### Primary References
+
+- OSINT Framework: https://osintframework.com
+- IntelTechniques: https://inteltechniques.com/tools
+- Bellingcat OSINT Guide: https://www.bellingcat.com/resources/how-tos/2023/04/10/open-source-intelligence-osint-tools
+- SANS OSINT Cheat Sheet: https://www.sans.org/blog/getting-started-with-open-source-intelligence
+
+### Credential Leak Databases
+
+- DeHashed: https://dehashed.com
+- HaveIBeenPwned: https://haveibeenpwned.com
+- IntelX: https://intelx.io
+- LeakIX: https://leakix.net
+- Snusbase: https://snusbase.com
+
+### Infrastructure Discovery
+
+- Shodan: https://shodan.io
+- Censys: https://censys.io
+- FOFA: https://fofa.info
+- ZoomEye: https://www.zoomeye.org
+- BinaryEdge: https://www.binaryedge.io
+- GreyNoise: https://www.greynoise.io
+
+### DNS and Certificate Intelligence
+
+- crt.sh: https://crt.sh
+- SecurityTrails: https://securitytrails.com
+- DNSdumpster: https://dnsdumpster.com
+- Passive DNS (CIRCL): https://www.circl.lu/services/passive-dns
+- RiskIQ (now Microsoft Defender TI): https://ti.defender.microsoft.com
+
+### Email Harvesting
+
+- Hunter.io: https://hunter.io
+- Phonebook.cz: https://phonebook.cz
+- EmailHippo: https://tools.emailhippo.com
+- Clearbit Connect: https://clearbit.com/resources/tools/connect
+
+### Social Media and Employee Enumeration
+
+- LinkedIn: https://linkedin.com
+- GitHub: https://github.com
+- Twitter/X Advanced Search: https://twitter.com/search-advanced
+- Facebook Graph: https://www.facebook.com/search
+- Pipl: https://pipl.com
+
+### Training and Practice
+
+- TryHackMe OSINT rooms: https://tryhackme.com/hacktivities?tab=search&value=osint
+- HackTheBox OSINT challenges: https://app.hackthebox.com
+- TraceLabs CTF (OSINT for missing persons): https://www.tracelabs.org
+
+---
+
+## Notes for Red Team Operators
+
+- Always confirm Rules of Engagement (RoE) before querying paid APIs against target data — some queries may be logged by the provider and visible to the target.
+- Certificate transparency and Shodan queries are fully passive and leave no trace on target infrastructure.
+- LinkedIn scraping may violate ToS — use dedicated accounts and rate-limit requests.
+- Leaked credential data must be handled according to engagement data handling policy — encrypt at rest, do not exfiltrate from secure environment.
+- GitHub secret scanning with TruffleHog should be run against all repos discovered in the target org, including forks and archived repos.
+- Employee enumeration output feeds directly into the `rt-phishing` and `rt-password-spray` skills.