opensquat 2.2__tar.gz → 2.2.2__tar.gz

opensquat-2.2.2/PKG-INFO

@@ -0,0 +1,249 @@
+Metadata-Version: 2.4
+Name: opensquat
+Version: 2.2.2
+Summary: openSquat - Detection of domain squatting, typosquatting, IDN homograph attacks, and phishing threats
+Author-email: Andre Tenreiro <andre@opensquat.com>
+License-Expression: GPL-3.0-only
+Project-URL: Homepage, https://opensquat.com
+Project-URL: Repository, https://github.com/atenreiro/opensquat
+Project-URL: Issues, https://github.com/atenreiro/opensquat/issues
+Keywords: cybersecurity,phishing,typosquatting,domain-squatting,osint,homograph,brand-protection
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Information Technology
+Classifier: Intended Audience :: System Administrators
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Security
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: confusable_homoglyphs>=2.0.0
+Requires-Dist: homoglyphs>=2.0.0
+Requires-Dist: colorama>=0.4.1
+Requires-Dist: requests>=2.20.0
+Requires-Dist: beautifulsoup4>=4.4.0
+Requires-Dist: dnspython>=2.0.0
+Dynamic: license-file
+
+<p align="center">
+  <img src="https://raw.githubusercontent.com/atenreiro/opensquat/master/screenshots/openSquat_logo.png" alt="openSquat Logo" width="400"/>
+</p>
+
+<h1 align="center">openSquat Core</h1>
+
+<p align="center">
+  <a href="#"><img src="https://img.shields.io/badge/python-3.8+-blue.svg" alt="Python 3.8+"></a>
+  <a href="https://github.com/atenreiro/opensquat/blob/master/LICENSE"><img src="https://img.shields.io/badge/License-GPLv3-blue.svg" alt="License: GPL v3"></a>
+  <a href="https://github.com/atenreiro/opensquat/issues"><img src="https://img.shields.io/github/issues/atenreiro/opensquat" alt="GitHub issues"></a>
+  <a href="https://github.com/atenreiro/opensquat/stargazers"><img src="https://img.shields.io/github/stars/atenreiro/opensquat" alt="GitHub stars"></a>
+</p>
+
+---
+
+## 📑 Table of Contents
+
+- [What is openSquat?](#-what-is-opensquat)
+- [Open-Core Model](#-open-core-model)
+- [Key Features](#-key-features)
+- [Quick Start](#-quick-start)
+- [Requirements](#-requirements)
+- [Usage](#-usage)
+- [Configuration](#%EF%B8%8F-configuration)
+- [Automation](#-automation)
+- [CLI Reference](#-cli-reference)
+- [Contributing](#-contributing)
+- [Author](#-author)
+- [License](#-license)
+
+---
+
+## 🎯 What is openSquat?
+
+openSquat is an **Open Source Intelligence (OSINT)** security tool that identifies cyber squatting threats targeting your brand or domains:
+
+| Threat Type | Description |
+|-------------|-------------|
+| 🎣 **Phishing** | Fraudulent domains mimicking your brand |
+| 🔤 **Typosquatting** | Domains with common typos (e.g., `gooogle.com`) |
+| 🌐 **IDN Homograph** | Look-alike characters from other alphabets |
+| 👥 **Doppelgänger** | Domains containing your brand name |
+| 🔀 **Bitsquatting** | Single-bit errors in domain names |
+
+## 🔓 Open-Core Model
+
+openSquat follows an **open-core model**:
+
+- **Core detection engine** — Open source and community-driven
+- **Advanced capabilities** — Delivered through commercial intelligence services
+
+This model enables transparency and community collaboration while supporting the scale, reliability, and operational requirements of enterprise use.
+
+---
+
+## ✨ Key Features
+
+- 📅 **Daily NRD feeds** — Automatic newly registered domain updates
+- 🔍 **Similarity detection** — Levenshtein distance algorithm
+- 🛡️ **VirusTotal integration** — Check domain reputation
+- 🌐 **Quad9 DNS validation** — Identify malicious domains
+- 📜 **Certificate Transparency** — Monitor SSL/TLS certificates
+- 📊 **Multiple output formats** — TXT, JSON, CSV
+
+---
+
+## 🚀 Quick Start
+
+### Install via pip (recommended)
+
+```bash
+pip install opensquat
+opensquat -k keywords.txt
+```
+
+### Or clone the repository
+
+```bash
+git clone https://github.com/atenreiro/opensquat
+cd opensquat
+pip install -r requirements.txt
+python opensquat.py -k keywords.txt
+```
+
+---
+
+## 📦 Requirements
+
+- **Python 3.8+**
+- Dependencies: `confusable_homoglyphs`, `homoglyphs`, `colorama`, `requests`, `dnspython`, `beautifulsoup4`
+
+---
+
+## 📖 Usage
+
+### Basic Commands
+
+```bash
+# Default run
+opensquat
+
+# Show all options
+opensquat -h
+
+# Use custom keywords file
+opensquat -k my_keywords.txt
+```
+
+### Validation Options
+
+```bash
+# DNS validation via Quad9
+opensquat --dns
+
+# Check Certificate Transparency logs
+opensquat --ct
+
+# Scan for open ports (80/443)
+opensquat --portcheck
+
+# Cross-reference phishing databases
+opensquat --phishing results.txt
+```
+
+### Output Formats
+
+```bash
+# Save as JSON
+opensquat -o results.json -t json
+
+# Save as CSV
+opensquat -o results.csv -t csv
+```
+
+### Confidence Levels
+
+| Level | Flag | Description |
+|-------|------|-------------|
+| 0 | `-c 0` | Very high (fewer results, high accuracy) |
+| 1 | `-c 1` | High (default) |
+| 2 | `-c 2` | Medium |
+| 3 | `-c 3` | Low |
+| 4 | `-c 4` | Very low (more results, more false positives) |
+
+---
+
+## ⚙️ Configuration
+
+### Keywords File (`keywords.txt`)
+
+```text
+# Lines starting with # are comments
+mycompany
+mybrand
+myproduct
+```
+
+### VirusTotal API Key (`vt_key.txt`)
+
+To use `--vt` or `--subdomains`, add your API key:
+```text
+# Get your free API key at https://www.virustotal.com
+your_api_key_here
+```
+
+---
+
+## 🤖 Automation
+
+Run daily via crontab:
+
+```bash
+# Every day at 8 AM (feeds update ~7:30 AM UTC)
+0 8 * * * /path/to/opensquat/opensquat.py -k keywords.txt -o results.json -t json
+```
+
+---
+
+## 📋 CLI Reference
+
+| Argument | Default | Description |
+|----------|---------|-------------|
+| `-k, --keywords` | `keywords.txt` | Keywords file to search |
+| `-o, --output` | `results.txt` | Output filename |
+| `-t, --type` | `txt` | Output format: `txt`, `json`, `csv` |
+| `-c, --confidence` | `1` | Confidence level (0-4) |
+| `-d, --domains` | — | Use local domain file instead of downloading |
+| `-u, --url` | opensquat feed | URL to download domain feed |
+| `--dns` | — | Enable Quad9 DNS validation |
+| `--doppelganger` | — | Doppelganger-only mode (keyword in domain + reachability check) |
+| `--ct` | — | Search Certificate Transparency logs |
+| `--phishing` | — | Cross-reference phishing database |
+| `--subdomains` | — | Fetch subdomains via VirusTotal |
+| `--portcheck` | — | Check for open ports 80/443 |
+| `--vt` | — | Validate against VirusTotal |
+
+---
+
+## 🤝 Contributing
+
+We welcome contributions! See our [Contributing Guide](CONTRIBUTING.md) for details.
+
+- 🐛 **Report bugs** via [GitHub Issues](https://github.com/atenreiro/opensquat/issues)
+- 💡 **Request features** by opening an issue
+- 🔧 **Submit PRs** for bug fixes or enhancements
+
+---
+
+## 👤 Author
+
+**Andre Tenreiro** — [LinkedIn](https://www.linkedin.com/in/andretenreiro/) · [PGP Key](https://mail-api.proton.me/pks/lookup?op=get&search=andre@opensquat.com)
+
+---
+
+## 📜 License
+
+This project is licensed under the [GNU GPL v3](LICENSE).

opensquat-2.2.2/README.md

@@ -0,0 +1,217 @@
+<p align="center">
+  <img src="https://raw.githubusercontent.com/atenreiro/opensquat/master/screenshots/openSquat_logo.png" alt="openSquat Logo" width="400"/>
+</p>
+
+<h1 align="center">openSquat Core</h1>
+
+<p align="center">
+  <a href="#"><img src="https://img.shields.io/badge/python-3.8+-blue.svg" alt="Python 3.8+"></a>
+  <a href="https://github.com/atenreiro/opensquat/blob/master/LICENSE"><img src="https://img.shields.io/badge/License-GPLv3-blue.svg" alt="License: GPL v3"></a>
+  <a href="https://github.com/atenreiro/opensquat/issues"><img src="https://img.shields.io/github/issues/atenreiro/opensquat" alt="GitHub issues"></a>
+  <a href="https://github.com/atenreiro/opensquat/stargazers"><img src="https://img.shields.io/github/stars/atenreiro/opensquat" alt="GitHub stars"></a>
+</p>
+
+---
+
+## 📑 Table of Contents
+
+- [What is openSquat?](#-what-is-opensquat)
+- [Open-Core Model](#-open-core-model)
+- [Key Features](#-key-features)
+- [Quick Start](#-quick-start)
+- [Requirements](#-requirements)
+- [Usage](#-usage)
+- [Configuration](#%EF%B8%8F-configuration)
+- [Automation](#-automation)
+- [CLI Reference](#-cli-reference)
+- [Contributing](#-contributing)
+- [Author](#-author)
+- [License](#-license)
+
+---
+
+## 🎯 What is openSquat?
+
+openSquat is an **Open Source Intelligence (OSINT)** security tool that identifies cyber squatting threats targeting your brand or domains:
+
+| Threat Type | Description |
+|-------------|-------------|
+| 🎣 **Phishing** | Fraudulent domains mimicking your brand |
+| 🔤 **Typosquatting** | Domains with common typos (e.g., `gooogle.com`) |
+| 🌐 **IDN Homograph** | Look-alike characters from other alphabets |
+| 👥 **Doppelgänger** | Domains containing your brand name |
+| 🔀 **Bitsquatting** | Single-bit errors in domain names |
+
+## 🔓 Open-Core Model
+
+openSquat follows an **open-core model**:
+
+- **Core detection engine** — Open source and community-driven
+- **Advanced capabilities** — Delivered through commercial intelligence services
+
+This model enables transparency and community collaboration while supporting the scale, reliability, and operational requirements of enterprise use.
+
+---
+
+## ✨ Key Features
+
+- 📅 **Daily NRD feeds** — Automatic newly registered domain updates
+- 🔍 **Similarity detection** — Levenshtein distance algorithm
+- 🛡️ **VirusTotal integration** — Check domain reputation
+- 🌐 **Quad9 DNS validation** — Identify malicious domains
+- 📜 **Certificate Transparency** — Monitor SSL/TLS certificates
+- 📊 **Multiple output formats** — TXT, JSON, CSV
+
+---
+
+## 🚀 Quick Start
+
+### Install via pip (recommended)
+
+```bash
+pip install opensquat
+opensquat -k keywords.txt
+```
+
+### Or clone the repository
+
+```bash
+git clone https://github.com/atenreiro/opensquat
+cd opensquat
+pip install -r requirements.txt
+python opensquat.py -k keywords.txt
+```
+
+---
+
+## 📦 Requirements
+
+- **Python 3.8+**
+- Dependencies: `confusable_homoglyphs`, `homoglyphs`, `colorama`, `requests`, `dnspython`, `beautifulsoup4`
+
+---
+
+## 📖 Usage
+
+### Basic Commands
+
+```bash
+# Default run
+opensquat
+
+# Show all options
+opensquat -h
+
+# Use custom keywords file
+opensquat -k my_keywords.txt
+```
+
+### Validation Options
+
+```bash
+# DNS validation via Quad9
+opensquat --dns
+
+# Check Certificate Transparency logs
+opensquat --ct
+
+# Scan for open ports (80/443)
+opensquat --portcheck
+
+# Cross-reference phishing databases
+opensquat --phishing results.txt
+```
+
+### Output Formats
+
+```bash
+# Save as JSON
+opensquat -o results.json -t json
+
+# Save as CSV
+opensquat -o results.csv -t csv
+```
+
+### Confidence Levels
+
+| Level | Flag | Description |
+|-------|------|-------------|
+| 0 | `-c 0` | Very high (fewer results, high accuracy) |
+| 1 | `-c 1` | High (default) |
+| 2 | `-c 2` | Medium |
+| 3 | `-c 3` | Low |
+| 4 | `-c 4` | Very low (more results, more false positives) |
+
+---
+
+## ⚙️ Configuration
+
+### Keywords File (`keywords.txt`)
+
+```text
+# Lines starting with # are comments
+mycompany
+mybrand
+myproduct
+```
+
+### VirusTotal API Key (`vt_key.txt`)
+
+To use `--vt` or `--subdomains`, add your API key:
+```text
+# Get your free API key at https://www.virustotal.com
+your_api_key_here
+```
+
+---
+
+## 🤖 Automation
+
+Run daily via crontab:
+
+```bash
+# Every day at 8 AM (feeds update ~7:30 AM UTC)
+0 8 * * * /path/to/opensquat/opensquat.py -k keywords.txt -o results.json -t json
+```
+
+---
+
+## 📋 CLI Reference
+
+| Argument | Default | Description |
+|----------|---------|-------------|
+| `-k, --keywords` | `keywords.txt` | Keywords file to search |
+| `-o, --output` | `results.txt` | Output filename |
+| `-t, --type` | `txt` | Output format: `txt`, `json`, `csv` |
+| `-c, --confidence` | `1` | Confidence level (0-4) |
+| `-d, --domains` | — | Use local domain file instead of downloading |
+| `-u, --url` | opensquat feed | URL to download domain feed |
+| `--dns` | — | Enable Quad9 DNS validation |
+| `--doppelganger` | — | Doppelganger-only mode (keyword in domain + reachability check) |
+| `--ct` | — | Search Certificate Transparency logs |
+| `--phishing` | — | Cross-reference phishing database |
+| `--subdomains` | — | Fetch subdomains via VirusTotal |
+| `--portcheck` | — | Check for open ports 80/443 |
+| `--vt` | — | Validate against VirusTotal |
+
+---
+
+## 🤝 Contributing
+
+We welcome contributions! See our [Contributing Guide](CONTRIBUTING.md) for details.
+
+- 🐛 **Report bugs** via [GitHub Issues](https://github.com/atenreiro/opensquat/issues)
+- 💡 **Request features** by opening an issue
+- 🔧 **Submit PRs** for bug fixes or enhancements
+
+---
+
+## 👤 Author
+
+**Andre Tenreiro** — [LinkedIn](https://www.linkedin.com/in/andretenreiro/) · [PGP Key](https://mail-api.proton.me/pks/lookup?op=get&search=andre@opensquat.com)
+
+---
+
+## 📜 License
+
+This project is licensed under the [GNU GPL v3](LICENSE).

{opensquat-2.2 → opensquat-2.2.2}/opensquat/__init__.py

@@ -1,4 +1,4 @@
 # -*- coding: utf-8 -*-
 # Module: __init__.py
 """openSquat Version."""
-__VERSION__ = "2.1.0"
+__VERSION__ = "2.2.2"

opensquat-2.2.2/opensquat/app.py

@@ -0,0 +1,160 @@
+# -*- coding: utf-8 -*-
+# Module: app.py
+"""
+openSquat
+
+(c) Andre Tenreiro
+
+* https://github.com/atenreiro/opensquat
+
+software licensed under GNU version 3
+"""
+import concurrent.futures
+import functools
+import io
+from colorama import Fore, Style
+
+from opensquat import file_input
+from opensquat.feed_manager import FeedManager
+from opensquat.dns_validator import DNSValidator
+from opensquat.squatting_detector import SquattingDetector
+
+
+class Domain:
+    """
+    Main orchestration class for OpenSquat.
+    """
+
+    def __init__(self):
+        """Initiator."""
+        self.domain_filename = None
+        self.keywords_filename = None
+        self.domain_total = 0
+        self.keywords_total = 0
+        self.list_domains = []
+        self.confidence_level = 2
+        self.doppelganger_only = False
+
+        self.feed_manager = None
+        self.dns_validator = None
+        self.squatting_detector = None
+
+        self.list_file_domains = []
+        self.list_file_keywords = []
+
+    def count_files(self):
+        (self.keywords_total, self.domain_total) = file_input.InputFile().main(
+            self.keywords_filename,
+            self.domain_filename
+        )
+
+    def read_files(self):
+        """
+        Method to read domain files
+        """
+        with open(self.domain_filename, mode='r') as file_domains:
+            for mydomains in file_domains:
+                domain = mydomains.replace("\n", "")
+                domain = domain.lower().strip()
+                # Skip comments and empty lines
+                if domain and not domain.startswith("#"):
+                    self.list_file_domains.append(domain)
+
+        with open(self.keywords_filename, mode='r') as file_keywords:
+            for line in file_keywords:
+                line = line.strip()
+                if line and not line.startswith("#"):
+                    self.list_file_keywords.append(line)
+
+    def print_info(self):
+        """
+        Method to print some configuration information.
+        """
+        print("[*] keywords:", self.keywords_filename)
+        print("[*] keywords total:", self.keywords_total)
+        print("[*] Total domains:", f"{self.domain_total:,}")
+
+        print("[*] Threshold:", self.squatting_detector.confidence.get(self.confidence_level, "unknown"))
+
+    @staticmethod
+    def verify_keyword_task(detector, domains_list, keyword_info):
+        """
+        Static worker method for parallel execution.
+        """
+        keyword, keyword_line_number, keywords_total = keyword_info
+
+        result_buffer = io.StringIO()
+        print(
+            f"[+] Starting Domain Squatting verification for '{keyword}' [{keyword_line_number}/{keywords_total}]",
+            file=result_buffer
+        )
+
+        print(
+            Fore.WHITE + "\n[*] Verifying keyword:",
+            keyword,
+            "[",
+            keyword_line_number,
+            "/",
+            keywords_total,
+            "]" + Style.RESET_ALL,
+            file=result_buffer
+        )
+
+        result_domains = detector.check(keyword, domains_list, result_buffer)
+        return result_buffer, result_domains
+
+    def worker(self):
+        """
+        Method that will compute all the similarity calculations between
+        the keywords and domain names.
+        """
+        with concurrent.futures.ProcessPoolExecutor() as executor:
+            keyword_infos = [
+                (keyword, i + 1, self.keywords_total)
+                for i, keyword in enumerate(self.list_file_keywords) if keyword
+            ]
+
+            worker_func = functools.partial(self.verify_keyword_task, self.squatting_detector, self.list_file_domains)
+
+            futs = [executor.submit(worker_func, k_info) for k_info in keyword_infos]
+
+        for fut in futs:
+            result_buffer, result_domains = fut.result()
+            print(result_buffer.getvalue())
+            self.list_domains.extend(result_domains)
+
+        return self.list_domains
+
+    def main(
+        self,
+        keywords_file,
+        confidence_level,
+        domains_file,
+        dns,
+        doppelganger_only=False,
+        feed_url="https://feeds.opensquat.com/opensquat-nrd-latest.txt",
+    ):
+        print("+---------- Checking Domain Squatting ----------+")
+        self.keywords_filename = keywords_file
+        self.domain_filename = domains_file
+        self.confidence_level = confidence_level
+        self.doppelganger_only = doppelganger_only
+
+        self.dns_validator = DNSValidator(use_dns=dns)
+        self.squatting_detector = SquattingDetector(
+            confidence_level=confidence_level,
+            doppelganger_only=doppelganger_only,
+            dns_validator=self.dns_validator
+        )
+
+        # Feed Management
+        self.feed_manager = FeedManager(feed_url=feed_url)
+        if self.domain_filename == "":
+            self.domain_filename = self.feed_manager.local_filename
+            self.feed_manager.ensure_feeds()
+
+        self.count_files()
+        self.read_files()
+        self.print_info()
+
+        return self.worker()