@lovehlmmm/primekit 1.0.0

package/skills/cti-expert/techniques/web-traffic-analysis.md

@@ -0,0 +1,180 @@
+# Web Traffic Analysis Module
+
+> **Module ID:** WEB-TRAF-001
+> **Version:** 1.0.0
+> **Phase:** 5 - Enhancement Modules
+> **Classification:** Educational Website Popularity & Traffic Estimation
+
+---
+
+## 1. Overview
+
+Estimates website traffic and popularity using free public ranking datasets and crawl-frequency signals. No single source is authoritative — this module triangulates across multiple lists to produce a weighted composite score and tier classification. Use for competitive research, site vetting, or assessing target infrastructure reach.
+
+---
+
+## 2. Tool Inventory
+
+| Priority | Source | Data Type | Free Limit | Notes |
+|----------|--------|-----------|-----------|-------|
+| Primary | Tranco | Rank list | Unlimited | research-oriented, aggregated |
+| Primary | Cloudflare Radar | Rank + trend | Unlimited | requires free CF account |
+| Primary | Cisco Umbrella Top 1M | DNS rank | Unlimited | daily CSV from S3 |
+| Secondary | CrUX API | Real user metrics | Unlimited | requires free Google API key |
+| Secondary | Majestic Million | Link-based rank | Unlimited | daily CSV download |
+| Tertiary | Wayback Machine CDX | Crawl frequency | Unlimited | no key required |
+| Tertiary | crt.sh | CT log frequency | Unlimited | no key required |
+
+---
+
+## 3. Investigation Workflow
+
+1. Normalize domain — strip `www.`, scheme, trailing slash
+2. Query primary sources (Tranco, Cloudflare, Umbrella) in parallel
+3. Query secondary sources (CrUX, Majestic) for corroboration
+4. Query tertiary signals (Wayback, crt.sh) for activity confirmation
+5. Apply scoring formula — compute weighted composite score (0–100)
+6. Assign tier from composite score
+7. Output report with per-source ranks, composite score, tier, and caveats
+
+---
+
+## 4. CLI Commands & Expected Output
+
+**Tranco rank:**
+```bash
+curl -s "https://tranco-list.eu/api/ranks/domain/<domain>" | jq .
+# Returns: {"domain":"example.com","ranks":[{"list":"2024-01-15","rank":1523}]}
+```
+
+**Cloudflare Radar:**
+```bash
+curl -s "https://api.cloudflare.com/client/v4/radar/ranking/domain/<domain>" \
+  -H "Authorization: Bearer <CF_FREE_TOKEN>" | jq '.result.details_0.bucket'
+# Returns bucket: top200 | top1k | top5k | top10k | top50k | top100k
+```
+
+**Cisco Umbrella Top 1M:**
+```bash
+curl -s "http://s3-us-west-1.amazonaws.com/umbrella-static/top-1m.csv.zip" -o /tmp/umbrella.zip
+unzip -p /tmp/umbrella.zip | grep -i "^[0-9]*,<domain>$"
+# Returns: rank,domain line or empty
+```
+
+**CrUX API:**
+```bash
+curl -s -X POST "https://chromeuxreport.googleapis.com/v1/records:queryRecord?key=<GOOGLE_KEY>" \
+  -H "Content-Type: application/json" \
+  -d '{"origin":"https://<domain>"}' | jq '.record.metrics | keys'
+# Presence of record = real Chrome user traffic exists
+```
+
+**Majestic Million:**
+```bash
+curl -s "https://downloads.majestic.com/majestic_million.csv" -o /tmp/majestic.csv
+grep -i ",<domain>," /tmp/majestic.csv | cut -d',' -f1,3
+# Returns: GlobalRank,Domain or empty
+```
+
+**Wayback Machine crawl frequency:**
+```bash
+curl -s "https://web.archive.org/cdx/search/cdx?url=<domain>/*&output=json&limit=0&showNumPages=true"
+# Returns page count; multiply by 100 for approximate snapshot count
+```
+
+**crt.sh CT log frequency:**
+```bash
+curl -s "https://crt.sh/?q=<domain>&output=json" | jq 'length'
+# Returns integer — certificate count; high = active domain
+```
+
+---
+
+## 5. Fallback Cascade
+
+| Primary Unavailable | Use Instead |
+|--------------------|-------------|
+| Tranco API down | Download weekly list CSV from tranco-list.eu manually |
+| Cloudflare token missing | Use Radar web UI at radar.cloudflare.com |
+| Umbrella S3 unavailable | Use Majestic Million as substitute (same weight) |
+| CrUX key missing | Skip; note in report as unscored |
+| Majestic CSV unavailable | Use crt.sh count as activity proxy |
+
+---
+
+## 6. Output Interpretation
+
+**Scoring methodology — weighted composite (0–100):**
+
+```
+Source        Weight   Score calculation
+----------    ------   --------------------------------------------------
+Tranco          25%    100 × (1 - rank/1_000_000),  0 if unranked
+Umbrella        25%    100 × (1 - rank/1_000_000),  0 if unranked
+Cloudflare      20%    top200=100, top1k=85, top5k=70, top10k=55,
+                       top50k=40, top100k=25, else=10
+CrUX            15%    100 if record exists, 0 if absent
+Majestic        15%    100 × (1 - rank/1_000_000),  0 if unranked
+
+Composite = sum(source_score × weight)
+```
+
+**Normalization formula:** `normalized = 100 × (1 - (rank - 1) / 999_999)`
+
+**Tier classification:**
+
+| Tier | Score | Description |
+|------|-------|-------------|
+| Mega | 85–100 | Top global properties |
+| Very Popular | 65–84 | Major site, broad audience |
+| Popular | 45–64 | Established niche or regional |
+| Moderate | 25–44 | Smaller but active site |
+| Niche | 10–24 | Limited audience / low traffic |
+| Unranked | 0–9 | No ranking signal found |
+
+**Tertiary signals:** Wayback snapshots > 500 = multi-year active history; crt.sh certs > 50 = active infrastructure.
+
+---
+
+## 7. Confidence Ratings
+
+| Finding | Confidence | Notes |
+|---------|-----------|-------|
+| Tranco rank (top 10k) | HIGH | Aggregated from multiple sources |
+| Cloudflare bucket (top 1k) | HIGH | Based on DNS resolver traffic |
+| CrUX record present | HIGH | Real Chrome user data |
+| Umbrella rank (top 100k) | MEDIUM | DNS-based, biased toward US |
+| Majestic rank | MEDIUM | Link-graph proxy, not visits |
+| Wayback frequency only | LOW | Crawl rate ≠ visitor traffic |
+| crt.sh count only | LOW | Infrastructure signal, not traffic |
+| Unranked in all sources | MEDIUM | Likely low traffic, not certainty |
+
+---
+
+## 8. Limitations
+
+- All sources are proxies (DNS queries, links, crawls) — none measure actual visits
+- **Geographic bias**: Umbrella skews US/enterprise; Cloudflare skews CF-proxied sites
+- **New domains**: Sites < 3 months old rarely appear in any ranking list
+- **CDN masking**: Shared CDN IPs may inherit CDN's rank, not the domain's own
+- **CrUX threshold**: < ~1000 monthly Chrome users → no record; absence ≠ zero traffic
+- **Majestic**: Measures inbound links; popular link farms can score falsely high
+- **Tranco lag**: Updated weekly; recent popularity changes not immediately reflected
+- **Cloudflare rate limits**: Free tier has undocumented soft limits; space queries 1–2 s apart
+
+---
+
+## 9. Command Reference
+
+| Command | Purpose | Input |
+|---------|---------|-------|
+| `/traffic <domain>` | Full composite traffic estimate | Domain name |
+| `/traffic <domain> --sources` | Per-source raw ranks | Domain name |
+| `/traffic <domain> --history` | Wayback + crt.sh activity signals | Domain name |
+
+Report output includes: per-source rank and normalized score, weighted composite (0–100), tier label, and tertiary activity signals (Wayback snapshot count, crt.sh cert count).
+
+---
+
+*Web Traffic Analysis Module v1.0.0*
+*Part of Free OSINT Expert Skill - Phase 5*

package/skills/cti-expert/techniques/whois-universal.md

@@ -0,0 +1,217 @@
+# Universal WHOIS Investigation
+
+> **Module ID:** WHOIS-UNI-001
+> **Version:** 1.0.0
+> **Phase:** Enhancement Module
+> **Classification:** Domain Registration & Ownership Intelligence
+
+---
+
+## 1. Overview
+
+Multi-TLD WHOIS with 4-layer fallback cascade. Covers gTLDs, ccTLDs (.vn, .th, .sg, .kr, etc.), and IP/ASN lookups. Zero API keys for core functionality.
+
+**When to use:** Any time domain ownership, registration timeline, or registrant identity is needed — especially for ccTLDs where basic `whois` fails.
+
+---
+
+## 2. WHOIS Cascade
+
+### Layer 1: whoisdomain Python Library (Primary)
+
+Auto-detects TLD → correct WHOIS server via IANA. ~90% coverage, parsed structured output.
+
+```bash
+# Install
+pip3 install whoisdomain
+
+# Query
+python3 -c "import whoisdomain; print(whoisdomain.query('example.vn'))"
+
+# Structured fields
+python3 -c "
+import whoisdomain
+d = whoisdomain.query('example.com')
+print(f'Registrant: {d.registrant}')
+print(f'Emails: {d.emails}')
+print(f'Created: {d.creation_date}')
+print(f'Expires: {d.expiration_date}')
+print(f'Name servers: {d.name_servers}')
+"
+```
+
+**Strengths:** Structured output, auto-TLD detection, no API key.
+**Weakness:** Some ccTLDs return raw text only.
+
+### Layer 2: CLI whois with TLD-Specific Server (Secondary)
+
+Direct server specification for ccTLDs. Use when whoisdomain fails to parse or connect.
+
+```bash
+whois -h whois.vnnic.vn domain.vn
+```
+
+#### ccTLD WHOIS Server Reference
+
+| TLD | WHOIS Server | Country |
+|-----|-------------|---------|
+| .vn | whois.vnnic.vn | Vietnam |
+| .th | whois.thnic.co.th | Thailand |
+| .sg | whois.sgnic.sg | Singapore |
+| .kr | whois.kr | South Korea |
+| .jp | whois.jprs.jp | Japan |
+| .cn | whois.cnnic.cn | China |
+| .tw | whois.twnic.net.tw | Taiwan |
+| .id | whois.id | Indonesia |
+| .my | whois.mynic.my | Malaysia |
+| .ph | whois.dot.ph | Philippines |
+| .in | whois.registry.in | India |
+| .ru | whois.tcinet.ru | Russia |
+| .br | whois.registro.br | Brazil |
+| .za | whois.registry.net.za | South Africa |
+| .ng | whois.nic.net.ng | Nigeria |
+| .ke | whois.kenic.or.ke | Kenya |
+| .de | whois.denic.de | Germany |
+| .fr | whois.nic.fr | France |
+| .it | whois.nic.it | Italy |
+| .es | whois.nic.es | Spain |
+| .nl | whois.sidn.nl | Netherlands |
+| .uk | whois.nic.uk | United Kingdom |
+| .au | whois.auda.org.au | Australia |
+| .nz | whois.srs.net.nz | New Zealand |
+| .mx | whois.mx | Mexico |
+| .ar | whois.nic.ar | Argentina |
+| .co | whois.nic.co | Colombia |
+
+**Usage:** `whois -h <server> <domain>`
+
+### Layer 3: Whoxy Free API (Tertiary)
+
+1595+ TLD coverage, JSON response. No auth, no published rate limits.
+
+```bash
+# Standard lookup
+curl -s "https://api.whoxy.com/?key=free&whois=domain.vn" | jq .
+
+# Reverse WHOIS by email
+curl -s "https://api.whoxy.com/?key=free&reverse=whois&email=target@email.com" | jq .
+
+# WHOIS history
+curl -s "https://api.whoxy.com/?key=free&history=domain.com" | jq .
+```
+
+**Strengths:** Massive TLD coverage, JSON output, reverse + history.
+**Weakness:** Third-party aggregator — may have stale data.
+
+### Layer 4: Web Scrape Fallback (Quaternary)
+
+When all programmatic methods fail:
+
+```bash
+# who.is web UI
+curl -sL "https://who.is/whois/domain.vn" | grep -A5 'Registrant'
+
+# Google for cached WHOIS
+# WebSearch: "domain.vn" whois registration registrant
+```
+
+---
+
+## 3. Reverse WHOIS (Free)
+
+Find all domains registered by same entity:
+
+```bash
+# Whoxy reverse by email
+curl -s "https://api.whoxy.com/?key=free&reverse=whois&email=target@email.com" | jq '.search_result[]'
+
+# Whoxy reverse by name
+curl -s "https://api.whoxy.com/?key=free&reverse=whois&name=John+Doe" | jq '.search_result[]'
+
+# ViewDNS reverse (WebSearch)
+# WebSearch: site:viewdns.info/reversewhois/?q=target@email.com
+
+# DomainBigData (WebSearch)
+# WebSearch: site:domainbigdata.com "registrant" "target@email.com"
+```
+
+---
+
+## 4. Historical WHOIS (Free)
+
+Track ownership changes over time:
+
+```bash
+# Whoxy history
+curl -s "https://api.whoxy.com/?key=free&history=domain.com" | jq '.whois_records[]'
+
+# Wayback Machine WHOIS snapshots
+# WebSearch: site:web.archive.org "domain.com" whois
+
+# Google cache of SecurityTrails/DomainTools pages
+# WebSearch: cache:securitytrails.com/domain/domain.com/dns
+```
+
+---
+
+## 5. IP/ASN WHOIS
+
+```bash
+# IP WHOIS (find network owner)
+whois 1.2.3.4
+# Key fields: NetName, OrgName, CIDR range, abuse contact
+
+# ASN lookup
+whois -h whois.radb.net AS12345
+# Or: https://bgp.tools/as/12345
+
+# ARIN (North America)
+curl -s "https://whois.arin.net/rest/ip/1.2.3.4" -H "Accept: application/json" | jq .
+
+# RIPE (Europe)
+curl -s "https://rest.db.ripe.net/search?query-string=1.2.3.4&source=ripe" -H "Accept: application/json" | jq .
+```
+
+---
+
+## 6. .vn Domain WHOIS Deep Dive
+
+Vietnam-specific WHOIS details:
+
+- **Server:** whois.vnnic.vn (port 43)
+- **Operator:** VNNIC (Vietnam Internet Network Information Center)
+- **Web interface:** https://vnnic.vn/en/whois-information
+- **Response fields:** registrant, admin contact, tech contact, name servers, dates
+- **Parsing notes:** Vietnamese text in registrant fields common; UTF-8 encoding
+- **No REST API available** — port 43 or web scrape only
+
+```bash
+# Direct query
+whois -h whois.vnnic.vn example.vn
+
+# If whois CLI unavailable, use netcat
+echo "example.vn" | nc whois.vnnic.vn 43
+```
+
+---
+
+## 7. Investigation Workflow
+
+1. `whoisdomain.query(domain)` → if structured result → **DONE**
+2. If parse fails → `whois -h <tld-server> domain` → raw text
+3. If connection refused → Whoxy API → JSON
+4. If all fail → web scrape who.is → HTML parse
+5. Cross-reference with DNS history for timeline
+
+Tag findings: `[whois-lib]` · `[whois-cli]` · `[whois-api]` · `[whois-scrape]`
+
+---
+
+## 8. Confidence Ratings
+
+| Source | Tag | Confidence | Notes |
+|--------|-----|-----------|-------|
+| whoisdomain parsed | [whois-lib] | HIGH | Direct WHOIS server query |
+| CLI whois raw | [whois-cli] | HIGH | Authoritative server |
+| Whoxy API | [whois-api] | MEDIUM | Third-party aggregator |
+| Web scrape | [whois-scrape] | LOW | May be cached/stale |

package/skills/cti-expert/techniques/wifi-ssid-osint.md

@@ -0,0 +1,221 @@
+# WiFi / SSID OSINT Module
+
+> **Module ID:** WIFI-OSINT-001
+> **Version:** 1.0.0
+> **Phase:** 5 - Enhancement Modules
+> **Classification:** Wireless Network Intelligence & Geolocation
+> **Source:** Adapted from [illwill/osint](https://gitlab.com/illwill/osint)
+
+---
+
+## 1. Overview
+
+WiFi/SSID OSINT maps wireless networks to physical locations using the Wigle.net wardriving database. Given a network name (SSID) or MAC address (BSSID), it returns GPS coordinates, encryption type, channel, timestamps, and surrounding infrastructure — enabling geolocation without GPS metadata.
+
+**Key use cases:** Physical location correlation, travel pattern analysis, rogue AP detection, proximity verification, device tracking through WiFi probe requests.
+
+---
+
+## 2. Tool Inventory
+
+### Primary — Wigle.net API v2 (free tier: 50 queries/day)
+```bash
+# Register free account at https://wigle.net/account
+# Get API name + token from https://wigle.net/account
+
+# Search by SSID
+curl -s -H "Authorization: Basic $(echo -n 'API_NAME:API_TOKEN' | base64)" \
+  "https://api.wigle.net/api/v2/network/search?ssid=TARGET_SSID"
+
+# Search by BSSID (MAC address)
+curl -s -H "Authorization: Basic $(echo -n 'API_NAME:API_TOKEN' | base64)" \
+  "https://api.wigle.net/api/v2/network/search?netid=AA:BB:CC:DD:EE:FF"
+
+# Search by location (bounding box)
+curl -s -H "Authorization: Basic $(echo -n 'API_NAME:API_TOKEN' | base64)" \
+  "https://api.wigle.net/api/v2/network/search?latrange1=37.7&latrange2=37.8&longrange1=-122.5&longrange2=-122.4"
+```
+
+### Secondary — Wigle Web UI (no API key needed for basic search)
+```
+https://wigle.net/search
+Manual browser search — limited results without account
+```
+
+### Tertiary — Google Geolocation API Fallback
+```bash
+# If BSSIDs known, Google can geolocate from WiFi access points
+curl -s -X POST "https://www.googleapis.com/geolocation/v1/geolocate?key=API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{"wifiAccessPoints":[{"macAddress":"AA:BB:CC:DD:EE:FF","signalStrength":-65}]}'
+```
+
+### Web Fallback — Search Engine Dorks
+```
+"TARGET_SSID" site:wigle.net
+"TARGET_SSID" wifi location OR coordinates
+"TARGET_BSSID" geolocation
+```
+
+---
+
+## 3. Investigation Workflow
+
+```
+Step 1: Identify target SSID or BSSID
+  └─ Source: device logs, packet captures, metadata, interview data
+  └─ Normalize BSSID to AA:BB:CC:DD:EE:FF format (uppercase, colon-separated)
+
+Step 2: Query Wigle.net API for matching networks
+  └─ Search by SSID first (may return multiple matches)
+  └─ Search by BSSID for exact AP match (unique per device)
+
+Step 3: Extract location data from results
+  └─ GPS coordinates (trilat/trilong)
+  └─ Address (country, region, city, road, postal code)
+  └─ Generate Google Maps link: https://maps.google.com/maps?q={lat},{long}
+
+Step 4: Analyze network metadata
+  └─ Encryption type reveals security posture (WPA3 vs WEP vs Open)
+  └─ Channel and frequency band (2.4GHz vs 5GHz)
+  └─ First/last seen timestamps reveal operational period
+
+Step 5: Cross-reference with other OSINT findings
+  └─ Match coordinates against known subject addresses
+  └─ Compare SSID naming patterns (corporate naming conventions, personal names)
+  └─ Check if same SSID appears at multiple locations (mobile hotspot indicator)
+
+Step 6: Assess confidence and document
+  └─ Multiple BSSID matches at same location = HIGH confidence
+  └─ Common SSID name with many matches = LOW confidence (e.g., "linksys")
+  └─ Single unique SSID match = MEDIUM confidence
+```
+
+---
+
+## 4. API Response Fields
+
+```json
+{
+  "trilat": 37.7749,        // GPS latitude
+  "trilong": -122.4194,     // GPS longitude
+  "ssid": "TargetNetwork",  // Network name
+  "netid": "AA:BB:CC:DD:EE:FF",  // MAC/BSSID
+  "encryption": "wpa2",     // Security type
+  "channel": 6,             // WiFi channel
+  "country": "US",
+  "region": "California",
+  "city": "San Francisco",
+  "road": "Market St",
+  "housenumber": "100",
+  "postalcode": "94105",
+  "firsttime": "2023-01-15T10:30:00",  // First observed
+  "lasttime": "2024-06-20T14:15:00"    // Last observed
+}
+```
+
+---
+
+## 5. Fallback Cascade
+
+```
+Wigle API key not configured?
+  └─> Use Wigle web UI (manual, limited results)
+  └─> Google dork: "SSID_NAME" site:wigle.net
+
+Wigle quota exhausted (50/day free)?
+  └─> Google Geolocation API (if BSSIDs known)
+  └─> Search: "BSSID" wifi geolocation
+
+SSID too common (100+ results)?
+  └─> Narrow by BSSID instead
+  └─> Filter by geographic region if approximate area known
+  └─> Cross-reference with other findings to disambiguate
+
+No results found?
+  └─> Network may not be wardrived yet (rural, private, new)
+  └─> Try vendor OUI lookup on first 3 octets of BSSID
+       https://api.macvendors.com/AA:BB:CC
+```
+
+---
+
+## 6. Output Interpretation
+
+### Encryption Significance
+```
+open      → No password; public hotspot or misconfigured AP
+wep       → Deprecated encryption; trivially broken; likely legacy device
+wpa       → Older WPA; vulnerable to dictionary attacks
+wpa2      → Standard enterprise/home security; common
+wpa3      → Latest standard; security-conscious deployment
+wpa2-ent  → Enterprise with RADIUS auth; corporate environment
+```
+
+### SSID Pattern Analysis
+```
+Corporate naming:     "ACME-Corp-5G", "Guest-ACME" → organizational affiliation
+Personal naming:      "John's iPhone", "SmithFamily" → identity leakage
+Default naming:       "NETGEAR-5G", "TP-Link_ABCD" → router vendor identification
+Hidden/no broadcast:  Empty SSID → security-aware operator
+Mobile hotspot:       Same SSID at multiple distant locations → device travels with owner
+```
+
+### Temporal Analysis
+```
+firsttime + lasttime close together → temporary deployment or brief observation
+firsttime old + lasttime recent → persistent installation
+firsttime recent + no lasttime → newly deployed
+Multiple entries, different locations → mobile AP (phone hotspot, vehicle)
+```
+
+---
+
+## 7. Confidence Ratings
+
+| Finding Type | Confidence | Notes |
+|---|---|---|
+| BSSID-matched location | HIGH | MAC address is unique per device |
+| Unique SSID location | MEDIUM | Unique names likely match one network |
+| Common SSID location | LOW | "linksys", "default" match thousands |
+| Encryption type | HIGH | Directly reported by scanner |
+| Owner identity from SSID | MEDIUM | SSID may contain names but could be spoofed |
+| Travel pattern from multi-location | MEDIUM | Requires BSSID match, not just SSID |
+| Vendor from OUI | HIGH | First 3 MAC octets = manufacturer |
+
+---
+
+## 8. Limitations
+
+1. **Coverage gaps** — Wigle depends on wardrivers; rural/private areas may have no data
+2. **API quota** — Free tier: 50 queries/day; register at wigle.net for account
+3. **SSID collisions** — Common network names produce many false positives
+4. **MAC randomization** — Modern devices randomize probe request MACs; AP BSSIDs are stable
+5. **Temporal decay** — Networks get replaced; old data may not reflect current state
+6. **Legal considerations** — Wardriving legality varies by jurisdiction; Wigle data is contributed voluntarily
+7. **GPS accuracy** — Coordinates reflect the wardriver's position, not the exact AP location (typically within 50m)
+
+---
+
+## 9. OPSEC Considerations
+
+- Wigle API queries are logged; your search patterns reveal investigative interest
+- Consider using Tor or VPN when querying Wigle API for sensitive investigations
+- SSID probing from your own device can expose your MAC address to nearby monitors
+- Registered Wigle accounts tie queries to your identity; use investigation-specific accounts
+
+---
+
+## 10. Command Reference
+
+| Command | Purpose | Input |
+|---|---|---|
+| `/wifi [ssid]` | Search Wigle for SSID geolocation | Network name |
+| `/wifi --bssid [mac]` | Search by exact MAC address | AA:BB:CC:DD:EE:FF |
+| `/wifi --area [lat,long,radius]` | Search networks in geographic area | Coordinates + radius |
+
+---
+
+*WiFi/SSID OSINT Module v1.0.0*
+*Part of CTI Expert Skill — adapted from illwill/osint (gitlab.com/illwill/osint)*
+*For authorized investigation and educational purposes only*